Here's what I've learned.

Picking our poison: The trade-offs of tracking

IP-based fingerprinting

Hash the voter's IP to restrict votes to one per IP.

Except not really. A university campus shares one public IP. An entire office behind a corporate NAT looks like a single user. You just locked out hundreds of legitimate voters. On the flip side, anyone with a VPN rotates their IP in seconds and votes again.

It works well enough for casual polls where the stakes are low. It falls apart the moment someone actually wants to game it.

Browser fingerprinting

Libraries like FingerprintJS build a hash from your screen resolution, installed fonts, GPU renderer, and timezone. Because this fingerprint relies entirely on your browser's hardware profile, it requires zero cookies and easily survives incognito mode.

Sounds clever, and it is. But two people on identical company laptops with the same OS image produce the same fingerprint. Browser updates change the hash. And privacy-focused browsers actively fight fingerprinting by randomising these signals. You're building on sand that shifts with every Chrome update.

Cookie / localStorage tracking

The simplest approach is to drop a cookie or localStorage flag after voting. Check it before allowing another vote. But it is the easiest to defeat.

Clearing cookies or opening an incognito window bypasses it instantly. Even non-technical users know this trick.

That said: most people won't bother. If your poll is "Where should we eat this Friday," cookies are fine.

Account-based authentication

Mandatory login to tie one vote to one account. This is the gold standard for accuracy, if someone has to authenticate with Google or GitHub, creating duplicate votes means creating duplicate accounts across those providers. That's real friction for an attacker.

The cost is real friction for everyone else too. Requiring login for a casual poll kills participation. Half your audience bounces at the login screen. The ones who stay are a self-selected group, which skews your results in a different way.

CAPTCHA (Cloudflare Turnstile, reCAPTCHA, hCaptcha)

CAPTCHAs don't prevent duplicate votes directly, rather they prevent bots. Cloudflare Turnstile is the nicest version of this, it proves a human is present, not that the human hasn't voted before.

You'd pair CAPTCHA with another method. CAPTCHA + IP hashing stops casual scripting via Postman or curl, the CAPTCHA token requires a browser environment, so raw API calls can't fake it. But a determined human can still solve the CAPTCHA repeatedly from different IPs.

Worth adding if bot spam is a realistic threat. Overkill for a team lunch poll.

Rate limiting

It helps to throttle submissions per IP, like allowing five votes per minute. This doesn't prevent duplicates, but it slows down brute-force attempts. It's a supporting measure, not a standalone solution.

The nice thing: it costs almost nothing to implement and catches the laziest attacks. The bad thing: it only catches the laziest attacks.

Device attestation (Apple DeviceCheck, Android SafetyNet)

Mobile platforms offer APIs that let you mark a device as "already voted" at the hardware level. The user can't clear it. Factory reset doesn't help.

Powerful, but platform-locked, and web polls can't use it. And it ties your system to Apple's and Google's infrastructure, which is a dependency you might not want.

How they compare

The Postman test

Here's the question nobody asks until it's too late: what happens when someone opens Postman, grabs your POST /api/poll/:id/respond endpoint, and starts firing requests?

Attacker with Postman / Curl / Python Script:

How each method holds up:

The uncomfortable truth about IP + UA hashing: it stops a lazy Postman test (same IP, same default UA = same fingerprint, duplicate rejected). But anyone who thinks to set a custom User-Agent header which is one line in Postman gets around it. The User-Agent is a client-supplied header. You're trusting the attacker to identify themselves honestly.

That's why CAPTCHA and auth are the only methods that truly block programmatic abuse. Everything else is a filter for laziness, not intent.

For Versus, I accepted this. The rate limiter catches the spray-and-pray scripts (5 req/min per IP), and the fingerprint index catches the ones that don't bother faking headers. A determined attacker with rotating IPs and random UAs could still stuff votes, but at that point, they're working harder than most people work at their actual jobs. For a poll about where to eat Friday, that's a risk I'll take.

What I actually used in Versus

I built a poll platform called Versus for a hackathon, and I needed something that worked for two very different modes: anonymous polls (anyone can vote) and authenticated polls (login required).

For authenticated polls, it's straightforward. Require OAuth login via Google or GitHub. Store respondent (user ID) on each response. A compound unique index on { poll, respondent } with a partialFilterExpression makes the database reject duplicates at the storage layer. One account with one vote is enforced by MongoDB. Can't be bypassed from Postman because you'd need a valid session cookie.

For anonymous polls, I went with IP + User-Agent hashing. When a vote comes in, I compute SHA-256(IP + User-Agent) and store it as a fingerprint on the response. Another compound unique index on { poll, fingerprint } rejects duplicates at the database level. The hash is one-way, and I never store raw IPs or user agents.

Why IP + UA instead of just IP? I had a realisation that my phone and laptop on the same WiFi share a public IP. Pure IP hashing meant my entire household got one vote per poll. Adding the User-Agent string means different devices (phone Safari vs. laptop Chrome) produce different hashes, even on the same network. The same browser on the same device still can't double-vote, but at least my family can each have an opinion.

I chose not to add CAPTCHA or browser fingerprinting just to keep it simple and flexible. The main threat is someone idly hitting refresh, not a coordinated bot attack. IP+UA hashing catches that. And the cost of false negatives (one person switching browsers to vote twice) is low.

If I needed higher assurance, I'd add Cloudflare Turnstile as a bot gate and push users toward authenticated mode.

How the rest of the industry does it

While building Versus, I studied what existing platforms actually do. The interesting thing: their anti-duplicate strategy almost always mirrors their business model.

Casual poll platforms optimise for participation. Survey tools optimise for response quality. Social platforms lean on account reputation. Enterprise tools optimise for auditability. And actual election systems treat accuracy as non-negotiable, friction be damned.

Casual polls (StrawPoll, Mentimeter, Poll Everywhere)

StrawPoll implements Cookies, IP tracking, optional CAPTCHA, rate limiting. They openly acknowledge that VPNs bypass their limits, and duplicate prevention is "best effort." Their goal is stopping casual spam.

Mentimeter and Poll Everywhere optimise for the "join instantly with a code" experience. Session limits, per-device restrictions, throttling, abuse detection. But they accept that a determined user can vote multiple times, because their use case is audience interaction during a live presentation. Locking down the poll harder than the room it's running in doesn't make sense.

Survey platforms (SurveyMonkey, Google Forms, Typeform)

SurveyMonkey's strongest mechanism is single-use survey links. Each respondent gets a unique URL (survey.com/r/xyz123token), and the token is invalidated after submission. For anonymous surveys, they fall back to cookies, IP restrictions, and session tracking. For controlled surveys, they have email invitations, unique links, authenticated respondents. The unique-link approach is extremely common in enterprise surveys.

Google Forms gives creators a toggle: "Limit to 1 response," which requires a Google account. Without it, there's essentially no strong duplicate prevention, they rely on cookies and Google's internal abuse detection. The hidden advantage Google has is that they already know enormous amounts about device reputation and traffic patterns. Small apps can't replicate that.

Typeform is interesting because they care more about response quality than strict uniqueness. They'll detect suspicious patterns and flag low-quality responses rather than hard-blocking. Cookies, hidden fields, optional auth, reCAPTCHA, and behavioural fraud systems for enterprise tiers.

Enterprise (Qualtrics)

This is where things get serious. Qualtrics supports SSO, signed links, panel management, fraud scoring, behavioural analytics, bot detection, geo analysis, and digital fingerprinting. They also do things like "speeding" detection (completed too fast to be human) and "straight-lining" detection (selecting the same option repeatedly). At this level, fraud analysis matters more than prevention, because sophisticated survey fraud is impossible to fully prevent.

Social media (X, Reddit)

X Polls has one vote per authenticated account. Internally, they likely layer account trust, age, phone verification, and spam reputation. But the visible enforcement is just "log in."

Reddit Polls: also account-based, but Reddit has an enormous hidden advantage, which is account reputation history. A 7-year-old active account is far more trustworthy than u/free_crypto_airdrop_19382. They can weight trust internally in ways that new platforms can't.

Actual voting systems (ElectionBuddy, Simply Voting)

Student government, union elections, shareholder voting use unique voter rolls, single-use cryptographic tokens, email verification, identity verification, audit logs, and cryptographic receipts. Some support anonymous-but-verifiable voting. The philosophy flips completely: accuracy over participation friction.

What the industry is converging toward

Pure IP blocking is considered weak now. CGNAT, mobile carriers, VPNs, IPv6 privacy rotation, an IP address is a weak signal, not an identity. Browser fingerprinting is weakening too. Safari ITP, Firefox anti-fingerprinting, Brave randomisation, Chrome's privacy sandbox direction. fingerprinting is now one signal among many, not the core system.

Most modern platforms are converging toward a layered model:

Versus sits squarely in the casual-poll tier and that's intentional. Low-friction anonymous mode, stronger authenticated mode, database-level uniqueness, rate limiting, and a pragmatic threat model. The biggest industry-standard addition I'm currently missing is probably a lightweight bot/risk layer like Cloudflare Turnstile, plus optional trust scoring over time.

But that's a problem I am looking forward to solve.

Versus is live here

This was a small project for hackathon, but I would be making some updates in near future. I welcome all the ideas, features and bug reports on GitHub.

Here is what a basic server looks like with just Node:

const http = require('http');

const server = http.createServer((req, res) => {
  if (req.method === 'GET' && req.url === '/hello') {
    res.writeHead(200, { 'Content-Type': 'application/json' });
    res.end(JSON.stringify({ message: 'hi there' }));
  } else {
    res.writeHead(404, { 'Content-Type': 'text/plain' });
    res.end('not found');
  }
});

server.listen(3000, () => {
  console.log('running on port 3000');
});

Two routes and the code is already getting annoying. Imagine adding fifteen more. Imagine adding POST body parsing to each one. Imagine doing error handling across all of them. I tried. I got about four routes deep before the if/else chain became unreadable.

Express exists because nobody wants to write that.

What Express actually is

Express is a library that sits on top of Node's http module. It does not replace it. Under the hood, Express still creates an http.Server. What it gives you is a cleaner way to define routes, read request data, and send responses without manually wiring everything together yourself.

Install it and set up a project:

mkdir my-express-app && cd my-express-app
npm init -y
npm install express

Now the same server from above, rewritten:

const express = require('express');
const app = express();

app.get('/hello', (req, res) => {
  res.json({ message: 'hi there' });
});

app.listen(3000, () => {
  console.log('running on port 3000');
});

No writeHead, or JSON.stringify. No checking req.method and req.url manually. One line defines the route, the method, and the handler. res.json() sets the content type and stringifies the object for you.

That difference gets wider the more routes you add.

The request comes in. Express checks its list of registered routes. If it finds a match for both the HTTP method and the URL path, it runs the handler function you gave it. If nothing matches, Express sends a 404 by default. You do not have to write that yourself.

Handling GET requests

GET requests are how clients ask for data. When you type a URL in your browser, that is a GET request. When a frontend calls fetch('/api/users'), that is a GET request.

const express = require('express');
const app = express();

const users = [
  { id: 1, name: 'Priya', email: 'priya@example.com' },
  { id: 2, name: 'Ravi', email: 'ravi@example.com' },
  { id: 3, name: 'Meera', email: 'meera@example.com' },
];

// get all users
app.get('/users', (req, res) => {
  res.json(users);
});

app.listen(3000, () => console.log('running on 3000'));

Hit http://localhost:3000/users in your browser and you get:

[
  { "id": 1, "name": "Priya", "email": "priya@example.com" },
  { "id": 2, "name": "Ravi", "email": "ravi@example.com" },
  { "id": 3, "name": "Meera", "email": "meera@example.com" }
]

Now say you want a single user by their ID. Express lets you put a colon in the URL path to create a route parameter:

// get one user by id
app.get('/users/:id', (req, res) => {
  const id = parseInt(req.params.id);
  const user = users.find(u => u.id === id);

  if (!user) {
    return res.status(404).json({ error: 'user not found' });
  }

  res.json(user);
});

req.params.id pulls the value from the URL. A request to /users/2 gives you req.params.id as the string "2". You parse it to a number, look it up, and either return the user or a 404. That :id syntax is called a route parameter, and you can have multiple in one path: /users/:userId/orders/:orderId would give you both req.params.userId and req.params.orderId.

One thing that tripped me up early: route parameters always come back as strings. Even if the URL says /users/2, req.params.id is "2", not 2. Forgetting to parse it leads to comparison bugs that are annoying to track down because "2" === 2 is false in JavaScript.

Handling POST requests

POST requests send data to the server. A signup form, a new order, a comment being submitted. The data lives in the request body, and Express does not parse it automatically. You need to tell it how.

app.use(express.json());

That one line is middleware. It tells Express to parse incoming JSON bodies and attach the result to req.body. Without it, req.body is undefined and you will stare at your code wondering why nothing works. I spent 20 minutes on this the first time. Add this line near the top of your file, before your route definitions.

const express = require('express');
const app = express();

app.use(express.json()); // parse JSON bodies

const users = [
  { id: 1, name: 'Priya', email: 'priya@example.com' },
  { id: 2, name: 'Ravi', email: 'ravi@example.com' },
];

let nextId = 3;

app.post('/users', (req, res) => {
  const { name, email } = req.body;

  if (!name || !email) {
    return res.status(400).json({ error: 'name and email are required' });
  }

  const newUser = { id: nextId++, name, email };
  users.push(newUser);

  res.status(201).json(newUser);
});

app.listen(3000, () => console.log('running on 3000'));

Test it with curl from your terminal:

curl -X POST http://localhost:3000/users \
  -H "Content-Type: application/json" \
  -d '{"name": "Aditya", "email": "aditya@example.com"}'

Response:

{ "id": 3, "name": "Aditya", "email": "aditya@example.com" }

The 201 status code means "created." You could send 200 and it would work fine, but 201 tells the client specifically that a new resource was created. Using the right status codes is a habit worth building early. 200 means OK. 201 means created. 400 means the client sent bad data. 404 means not found. 500 means the server broke. Most real APIs use these consistently, and your code becomes easier to debug when the status code already tells you what category of problem you are looking at.

Here is what the full flow looks like when a POST request comes in:

Sending different types of responses

res.json() is the one you will use most, but Express has a few other response methods worth knowing.

// send plain text
app.get('/health', (req, res) => {
  res.send('OK');
});

// send JSON with a status code
app.get('/status', (req, res) => {
  res.status(200).json({ status: 'running', uptime: process.uptime() });
});

// send just a status code with no body
app.delete('/users/:id', (req, res) => {
  // ... delete logic here
  res.sendStatus(204); // 204 means "no content" - deleted successfully, nothing to return
});

// redirect to another URL
app.get('/old-page', (req, res) => {
  res.redirect('/new-page');
});

res.send() is smart about what you give it. Pass a string, it sets the content type to text/html. Pass an object, it behaves like res.json(). Pass a Buffer, it sets the type to application/octet-stream. I would recommend being explicit and using res.json() for objects and res.send() for strings, rather than relying on the auto-detection. It makes your intent obvious when someone else reads the code later.

One gotcha: do not call res.json() or res.send() twice in the same handler. Express will throw an error that says "Cannot set headers after they are sent to the client." This usually happens when you forget a return statement before sending an error response:

// broken - sends two responses
app.get('/users/:id', (req, res) => {
  const user = users.find(u => u.id === parseInt(req.params.id));

  if (!user) {
    res.status(404).json({ error: 'not found' });
    // missing return! code keeps running
  }

  res.json(user); // crashes here because a response was already sent
});

// fixed
app.get('/users/:id', (req, res) => {
  const user = users.find(u => u.id === parseInt(req.params.id));

  if (!user) {
    return res.status(404).json({ error: 'not found' }); // return stops execution
  }

  res.json(user);
});

That return before res.status(404) is easy to forget and hard to debug if you do not know what the error message means.

Putting it all together

Here is a complete working server with GET and POST routes, input validation, proper status codes, and the JSON parsing middleware. You can copy this into a file called server.js and run it with node server.js:

const express = require('express');
const app = express();
const PORT = 3000;

app.use(express.json());

const users = [
  { id: 1, name: 'Priya', email: 'priya@example.com' },
  { id: 2, name: 'Ravi', email: 'ravi@example.com' },
];

let nextId = 3;

// get all users
app.get('/users', (req, res) => {
  res.json(users);
});

// get single user
app.get('/users/:id', (req, res) => {
  const user = users.find(u => u.id === parseInt(req.params.id));
  if (!user) return res.status(404).json({ error: 'user not found' });
  res.json(user);
});

// create a user
app.post('/users', (req, res) => {
  const { name, email } = req.body;
  if (!name || !email) {
    return res.status(400).json({ error: 'name and email are required' });
  }
  const newUser = { id: nextId++, name, email };
  users.push(newUser);
  res.status(201).json(newUser);
});

// health check
app.get('/health', (req, res) => {
  res.send('OK');
});

app.listen(PORT, () => {
  console.log(`server running on http://localhost:${PORT}`);
});

Test it with these curl commands:

# get all users
curl http://localhost:3000/users

# get one user
curl http://localhost:3000/users/1

# create a user
curl -X POST http://localhost:3000/users \
  -H "Content-Type: application/json" \
  -d '{"name": "Aditya", "email": "aditya@example.com"}'

# try creating without required fields
curl -X POST http://localhost:3000/users \
  -H "Content-Type: application/json" \
  -d '{"name": "Aditya"}'

A few habits worth picking up now

Use nodemon during development. It restarts your server automatically when you save a file. Without it you will be pressing Ctrl+C and typing node server.js hundreds of times a day.

npm install --save-dev nodemon

Add a script to your package.json:

{
  "scripts": {
    "dev": "nodemon server.js"
  }
}

Then run npm run dev instead of node server.js.

Keep your route handlers short. If a handler is doing validation, database queries, and sending emails all in one function, it gets painful to maintain fast. As your app grows, pull logic into separate functions or files. You do not need to do this on day one, but keep it in mind as the file gets longer.

Always validate incoming data. Never trust req.body to contain what you expect. Check for missing fields. Check types when it matters. A missing return after sending an error response will crash your server, and bad input from a client should never be able to do that.

What comes next

This covers the basics: setting up Express, handling GET and POST, sending responses, and avoiding the common early mistakes. The next piece you will run into is middleware, which is how Express handles things like authentication, logging, and error handling across routes without duplicating code everywhere. That will be its own post.

For now, build something small. A to-do list API. A bookmarks server. Something where you have two or three routes that create and read data. The concepts stick faster when you are solving a problem you actually care about, even a tiny one.

References

• Express.js official docs

• Express routing guide

• Express req and res API reference

• MDN: HTTP status codes

• nodemon on npm

• Node.js http module docs

The problem was obvious once someone pointed it out: I was saving files to the server's local filesystem, and the hosting platform wiped the disk on every deploy. I had never actually thought about where uploaded files end up, or why that matters.

The two places files can live

When a user uploads a file to your Express app, it has to go somewhere. You have two real options: save it on the same machine running your server, or send it to an external storage service.

Local storage means writing the file to a folder on your server's disk. The file sits right there next to your code. You can open it in your terminal, check its size, delete it manually. It is simple and fast for development.

External storage means sending the file to a service like AWS S3, Google Cloud Storage, or Cloudinary. The file lives on their servers, not yours. You get back a URL. Your server never holds the file long term.

Local storage:
  User uploads photo -> Express saves to /uploads/photo.jpg -> served from your disk

External storage:
  User uploads photo -> Express sends to S3 -> S3 returns a URL -> you save that URL in your database

For learning and local development, saving to disk is fine. For anything you deploy, external storage is almost always the right call. Your server's disk can fill up, get wiped on redeploy, or just stop existing if the hosting provider recycles your instance. S3 does not have these problems.

I will focus on local storage for the rest of this post because that is what you need to understand first. External storage is a topic for later, and it only clicks once you understand what local storage actually does and where it falls short.

Where files land when you use Multer

Multer is the standard library for handling file uploads in Express. You have probably seen it in tutorials already. When a user submits a form with a file, Multer grabs that file from the incoming request and writes it to a destination you specify.

const express = require('express');
const multer = require('multer');
const path = require('path');

const storage = multer.diskStorage({
  destination: function (req, file, cb) {
    cb(null, 'uploads/');
  },
  filename: function (req, file, cb) {
    const uniqueName = Date.now() + '-' + Math.round(Math.random() * 1e9);
    cb(null, uniqueName + path.extname(file.originalname));
  },
});

const upload = multer({ storage });

const app = express();

app.post('/upload', upload.single('avatar'), (req, res) => {
  console.log(req.file);
  // {
  //   fieldname: 'avatar',
  //   originalname: 'selfie.png',
  //   filename: '1717012345678-482947123.png',
  //   path: 'uploads/1717012345678-482947123.png',
  //   size: 204800
  // }
  res.json({ message: 'uploaded', file: req.file.filename });
});

The destination function tells Multer which folder to write into. The filename function controls what the file gets named on disk. I am generating a unique name with a timestamp and random number because if two users both upload photo.jpg, the second one would overwrite the first. That is a bug you will not notice until a user complains that their profile picture is someone else's face.

The folder structure after a few uploads looks like this:

That uploads/ folder does not create itself. You need to make it before running the server, or Multer throws an error. A quick way to handle this:

const fs = require('fs');

const uploadDir = './uploads';
if (!fs.existsSync(uploadDir)) {
  fs.mkdirSync(uploadDir, { recursive: true });
}

Put that near the top of your server file, before Multer initializes.

Serving uploaded files so the browser can actually see them

Saving a file to uploads/ is only half the job. If a user uploads a profile picture, they need to see it in the browser. Right now, if you try to visit http://localhost:3000/uploads/1717012345678-482947123.png, Express returns a 404. The file is on disk but Express does not know you want to serve it.

Express does not serve files from your project folders by default. This is a good thing. You do not want someone guessing a URL and downloading your .env file or your server.js. Express only serves what you explicitly tell it to serve.

The way you tell it is express.static():

app.use('/uploads', express.static('uploads'));

That single line says: when someone requests a URL starting with /uploads, look for a matching file in the uploads/ folder on disk and send it back. Now http://localhost:3000/uploads/1717012345678-482947123.png returns the actual image.

The flow looks like this:

You can use any URL prefix you want. If you use app.use('/files', express.static('uploads')), the browser URL becomes /files/1717012345678-482947123.png even though the folder on disk is still called uploads. The URL prefix and the folder name do not have to match, though keeping them the same is less confusing.

A common pattern is serving the uploaded file URL from your API and storing it in your database:

app.post('/upload', upload.single('avatar'), async (req, res) => {
  const fileUrl = `/uploads/${req.file.filename}`;

  await User.findByIdAndUpdate(req.user._id, { avatar: fileUrl });

  res.json({ avatar: fileUrl });
});

On the frontend, you render it as:

<img src="/uploads/1717012345678-482947123.png" alt="User avatar" />

That is the entire chain from upload to display.

The security stuff you should not skip

File uploads are one of the most common attack vectors in web applications. Skipping these checks is how people end up with malware on their servers or their entire uploads folder exposed.

Check the file type. Multer has a fileFilter option that lets you reject files before they are saved. Do not trust the file extension alone. A file named cute-cat.jpg can contain executable code if someone renamed it. Check the MIME type too:

const upload = multer({
  storage,
  fileFilter: function (req, file, cb) {
    const allowed = ['image/jpeg', 'image/png', 'image/webp'];
    if (allowed.includes(file.mimetype)) {
      cb(null, true);
    } else {
      cb(new Error('Only JPEG, PNG, and WebP images are allowed'));
    }
  },
  limits: {
    fileSize: 5 * 1024 * 1024, // 5 MB
  },
});

The limits option caps file size. Without it, someone can upload a 2GB file and fill your disk. Your server stops responding, all users are affected, and you are debugging at midnight wondering why apt update fails because the disk is full.

Never use the original filename. I showed the random name approach earlier, and there is a real reason for it beyond avoiding collisions. If you save files with the original name, a user can upload a file called ../../server.js and potentially overwrite your actual server code. This is called a path traversal attack. The generated filename with Date.now() and a random number removes this risk entirely because the name has no directory separators and no meaningful path components.

Do not serve your project root as static. I have seen beginners write app.use(express.static('.')) to serve "everything" and wonder why their .env file is accessible from a browser. Only point express.static() at a dedicated upload folder that contains nothing sensitive.

Add authentication where it matters. If uploads should only be visible to the user who uploaded them, do not serve them with express.static() at all. Instead, write a route that checks permissions before sending the file:

app.get('/my-files/:filename', authMiddleware, async (req, res) => {
  const file = await File.findOne({
    filename: req.params.filename,
    owner: req.user._id,
  });

  if (!file) return res.status(404).json({ error: 'Not found' });

  res.sendFile(path.resolve('uploads', file.filename));
});

This way, a user cannot access another user's files by guessing the filename. The route checks ownership first.

A quick note on what happens when you deploy

When you push code to services like Railway, Render, or Heroku, the filesystem is ephemeral. Files saved to uploads/ survive until the next deploy or the next server restart, and then they are gone. This is not a bug. The platform provisions a fresh container each time.

That is the point where local storage stops being enough and external storage like S3 starts making sense. You store the file somewhere permanent, save the returned URL in your database, and your server never worries about disk space or lost files again. I will cover that setup in a later post. For now, knowing that this limitation exists saves you the confusion I had when my uploaded images all disappeared on deploy.

References

• Multer documentation on npm

• Express static files documentation

• OWASP file upload cheat sheet

• Express res.sendFile documentation

• AWS S3 documentation (for when you are ready for external storage)

That line is middleware. And once I understood what middleware is, a lot of Express stopped feeling like magic and started making sense.

What middleware actually is

When a request hits your Express server, it does not go straight to the route handler. It passes through a series of functions first. Each of those functions can look at the request, modify it, send a response, or pass control to the next function in line. Those functions are middleware.

Think of it like an airport. You do not walk off the plane and straight into the city. You go through passport control, then customs, then baggage claim. Each checkpoint can let you through, send you back, or redirect you somewhere else. Your route handler is the exit door. Middleware is everything between landing and getting outside.

A middleware function in Express looks like this:

function myMiddleware(req, res, next) {
  // do something with req or res
  next(); // pass control to the next middleware or route
}

Three parameters. req is the incoming request object. res is the response object. next is a function that, when called, moves execution to the next middleware in the chain. If you do not call next() and you do not send a response, the request just hangs. The client waits forever. No error, no timeout message from Express, nothing. It just sits there. This will trip you up at least once.

Where middleware sits in the request lifecycle

Every HTTP request that reaches your Express app follows the same path. It enters the app, passes through middleware functions in the order they were registered, hits a matching route handler (if one exists), and a response goes back to the client.

Client sends request
       |
       v
  Express app receives it
       |
       v
  Middleware 1 (e.g., parse JSON body)
       |
       v
  Middleware 2 (e.g., log the request)
       |
       v
  Middleware 3 (e.g., check authentication)
       |
       v
  Route handler (your actual logic)
       |
       v
  Response sent back to client

The order matters. If your authentication middleware is registered after your route handler, the route runs without any auth check. Express processes middleware in the exact order you call app.use() or app.get() in your code. Move a line up or down and the behavior changes. This is not a design quirk. It is how the whole system works.

A minimal example to see it moving

Before getting into categories and theory, here is a working Express app with two middleware functions and one route. Run it locally and watch the console.

const express = require('express');
const app = express();

// Middleware 1: log every request
app.use((req, res, next) => {
  console.log(`\({req.method} \){req.url}`);
  next();
});

// Middleware 2: add a timestamp to the request object
app.use((req, res, next) => {
  req.requestTime = new Date().toISOString();
  next();
});

// Route handler
app.get('/', (req, res) => {
  res.json({ message: 'hello', time: req.requestTime });
});

app.listen(3000, () => console.log('running on 3000'));

Hit http://localhost:3000/ in your browser or with curl. The console prints something like:

GET /

And the response body looks like:

{
  "message": "hello",
  "time": "2025-01-15T10:23:45.123Z"
}

Middleware 1 logged the method and URL. Middleware 2 attached a timestamp to req. The route handler read that timestamp and sent it back. Each function did one thing, then called next() to let the next one run. That is the entire pattern.

Types of middleware

Express middleware falls into a few categories based on where and how you register it.

Application-level middleware

This is middleware registered on the app object using app.use() or app.METHOD(). It runs for every matching request, or for every request if no path is specified.

// runs for ALL requests to any route
app.use((req, res, next) => {
  console.log('this runs on every request');
  next();
});

// runs only for requests to /api/users
app.use('/api/users', (req, res, next) => {
  console.log('this only runs for /api/users routes');
  next();
});

The first version with no path argument is the most common. You see it for things like body parsing, CORS headers, and logging. The second version with a path lets you scope middleware to a specific section of your API without affecting other routes.

Router-level middleware

Express has a Router class that works like a mini-app. You can attach middleware to a router instead of the main app, and the middleware only applies to routes defined on that router.

const express = require('express');
const router = express.Router();

// this middleware only applies to routes on this router
router.use((req, res, next) => {
  console.log('admin router middleware');
  next();
});

router.get('/dashboard', (req, res) => {
  res.json({ page: 'admin dashboard' });
});

router.get('/users', (req, res) => {
  res.json({ page: 'manage users' });
});

// mount the router at /admin
app.use('/admin', router);

Now the middleware only runs when someone hits /admin/dashboard or /admin/users. A request to /api/products never touches it. This is how larger Express apps stay organized. You group related routes into routers, attach the middleware those routes need, and mount the router at a path. The main app file stays clean.

In a real project the file structure usually looks something like this:

routes/
  admin.js      <-- router with admin-only middleware
  auth.js       <-- router for login/signup
  orders.js     <-- router for order CRUD
server.js       <-- mounts all routers with app.use()

Built-in middleware

Express ships with three built-in middleware functions. You have probably used at least one without knowing it was middleware.

express.json() parses incoming JSON request bodies. Without it, req.body is undefined on POST and PUT requests. This is the one that got me at the start.

app.use(express.json());

app.post('/users', (req, res) => {
  console.log(req.body); // { name: "Priya", email: "priya@example.com" }
  res.json({ received: true });
});

Without that express.json() line:

app.post('/users', (req, res) => {
  console.log(req.body); // undefined
  res.json({ received: true });
});

No error. No warning. Just undefined. You find out when your database insert fails because you passed undefined as the document.

express.urlencoded({ extended: true }) does the same thing for form submissions. HTML forms send data in a format called application/x-www-form-urlencoded, and this middleware parses that into req.body.

app.use(express.urlencoded({ extended: true }));

The extended: true option uses the qs library for parsing, which supports nested objects in form data. With extended: false it uses the built-in querystring module, which only handles flat key-value pairs. For most apps, extended: true is what you want.

express.static() serves files from a directory. CSS, images, JavaScript files for the browser. You point it at a folder and Express serves everything in it.

app.use(express.static('public'));

A file at public/style.css becomes accessible at http://localhost:3000/style.css. No route handler needed.

Execution order and why it matters more than you think

I mentioned earlier that order matters. Here is a concrete example of what goes wrong when you get it backwards.

const express = require('express');
const app = express();

// Route defined BEFORE the JSON parser
app.post('/users', (req, res) => {
  console.log(req.body);  // undefined
  res.json({ user: req.body });
});

// JSON parser registered AFTER the route
app.use(express.json());

app.listen(3000);

Send a POST request with a JSON body to /users. req.body is undefined. The JSON parser exists in your code, but Express already matched the route and ran the handler before reaching the parser. By the time express.json() is registered, the request is already done.

Fix:

const express = require('express');
const app = express();

// Parser FIRST
app.use(express.json());

// Route SECOND
app.post('/users', (req, res) => {
  console.log(req.body);  // { name: "Priya" }
  res.json({ user: req.body });
});

app.listen(3000);

The general rule: put app.use() middleware at the top of your file, before any route definitions. Body parsers, CORS, logging, authentication checks. All of it goes above your routes. There are exceptions (error handlers go at the bottom, which I will get to), but this ordering keeps things predictable.

When you have multiple middleware, Express runs them in sequence:

app.use(middlewareA);
app.use(middlewareB);
app.use(middlewareC);

app.get('/test', handler);

A GET request to /test runs middlewareA, then middlewareB, then middlewareC, then handler. Each one has to call next() for the next one to run. If middlewareB does not call next(), middlewareC and handler never execute.

The next() function

next() is what connects the chain. Without it, the pipeline stops. There are three things you can do with it.

Call it with no arguments to pass control to the next middleware:

app.use((req, res, next) => {
  console.log('step 1');
  next(); // move to step 2
});

Do not call it and send a response instead, which ends the cycle:

app.use((req, res, next) => {
  if (!req.headers.authorization) {
    return res.status(401).json({ error: 'No token provided' });
    // next() is never called. The chain stops here.
  }
  next();
});

Call it with an error to skip to the error-handling middleware:

app.use((req, res, next) => {
  try {
    // some operation that might throw
    const data = JSON.parse(req.headers['x-custom-data']);
    req.customData = data;
    next();
  } catch (err) {
    next(err); // jumps to the error handler
  }
});

Calling next(err) with an argument skips every regular middleware and route handler and goes straight to the first error-handling middleware. I will show what that looks like in a moment.

A common mistake: calling next() and then continuing to execute code after it.

app.use((req, res, next) => {
  if (!req.headers.authorization) {
    res.status(401).json({ error: 'unauthorized' });
    // forgot to return here
  }
  next(); // this still runs even after sending the 401
});

This causes the "Cannot set headers after they are sent to the client" error. The 401 response was already sent, then next() called the route handler which tried to send another response. The fix is either using return before next() in the else case, or adding return before the res.status() call:

app.use((req, res, next) => {
  if (!req.headers.authorization) {
    return res.status(401).json({ error: 'unauthorized' });
  }
  next();
});

That return prevents any code below it from running after the response is sent.

Real-world examples

Request logging

In development you want to see every request that hits your server. What method, what URL, how long it took. A logging middleware handles this without touching any route.

app.use((req, res, next) => {
  const start = Date.now();

  // this runs after the response is sent
  res.on('finish', () => {
    const duration = Date.now() - start;
    console.log(`\({req.method} \){req.url} \({res.statusCode} \){duration}ms`);
  });

  next();
});

Output when you hit a few routes:

GET / 200 3ms
GET /api/users 200 12ms
POST /api/users 201 8ms
GET /api/users/999 404 2ms

The res.on('finish') listener fires after Express sends the response, so you get the actual status code and timing. Without that listener, you would have to log before the response was sent and you would not know the status code yet.

In production, most people use morgan instead of writing their own logger. It gives you configurable log formats and can write to files:

const morgan = require('morgan');
app.use(morgan('dev'));

GET / 200 2.540 ms
POST /api/users 201 15.230 ms

But writing your own first, like the example above, teaches you what morgan is doing under the hood.

Authentication

A common pattern in APIs: certain routes require the user to be logged in. Instead of checking authentication inside every route handler, you write a middleware that does it once.

const jwt = require('jsonwebtoken');

function authenticate(req, res, next) {
  const header = req.headers.authorization;

  if (!header || !header.startsWith('Bearer ')) {
    return res.status(401).json({ error: 'No token provided' });
  }

  const token = header.split(' ')[1];

  try {
    const decoded = jwt.verify(token, process.env.JWT_SECRET);
    req.user = decoded; // attach user info to the request
    next();
  } catch (err) {
    return res.status(401).json({ error: 'Invalid or expired token' });
  }
}

You can apply it to specific routes:

// public routes, no auth needed
app.post('/api/login', loginHandler);
app.post('/api/signup', signupHandler);

// protected routes
app.get('/api/profile', authenticate, (req, res) => {
  // req.user exists here because authenticate put it there
  res.json({ user: req.user });
});

app.get('/api/orders', authenticate, (req, res) => {
  // same, req.user is available
  res.json({ orders: getUserOrders(req.user.id) });
});

Or apply it to an entire router:

const protectedRouter = express.Router();
protectedRouter.use(authenticate); // every route on this router requires auth

protectedRouter.get('/profile', profileHandler);
protectedRouter.get('/orders', ordersHandler);
protectedRouter.put('/settings', settingsHandler);

app.use('/api', protectedRouter);

The second approach is cleaner when you have many protected routes. You do not have to remember to add authenticate to each new route. If a route is on the protected router, it is protected.

A note about storing secrets: process.env.JWT_SECRET comes from a .env file loaded with dotenv. Do not hardcode the secret in your source code. A hardcoded secret that gets committed to Git is a leaked secret.

Request validation

You want to make sure the data coming into your API makes sense before your route handler tries to use it. Checking inside the handler works, but it gets repetitive fast. A validation middleware keeps the checking separate from the business logic.

Here is a simple version without any library:

function validateUser(req, res, next) {
  const { name, email } = req.body;
  const errors = [];

  if (!name || typeof name !== 'string' || name.trim().length < 2) {
    errors.push('Name is required and must be at least 2 characters');
  }

  if (!email || !/^\S+@\S+\.\S+$/.test(email)) {
    errors.push('A valid email is required');
  }

  if (errors.length > 0) {
    return res.status(400).json({ errors });
  }

  next();
}

app.post('/api/users', validateUser, (req, res) => {
  // if we reach here, name and email are valid
  const user = createUser(req.body);
  res.status(201).json(user);
});

Send a bad request:

curl -X POST http://localhost:3000/api/users \
  -H "Content-Type: application/json" \
  -d '{"name": "", "email": "not-an-email"}'

{
  "errors": [
    "Name is required and must be at least 2 characters",
    "A valid email is required"
  ]
}

The route handler never runs. The validation middleware caught the bad input and sent back a 400 before the request got any further.

For production apps, most teams use a validation library like Joi or Zod. They give you a schema-based approach that handles edge cases you do not want to think about manually:

const Joi = require('joi');

const userSchema = Joi.object({
  name: Joi.string().trim().min(2).max(60).required(),
  email: Joi.string().email().required(),
  age: Joi.number().integer().min(16).max(120),
});

function validate(schema) {
  return (req, res, next) => {
    const { error } = schema.validate(req.body, { abortEarly: false });
    if (error) {
      const messages = error.details.map(d => d.message);
      return res.status(400).json({ errors: messages });
    }
    next();
  };
}

app.post('/api/users', validate(userSchema), (req, res) => {
  const user = createUser(req.body);
  res.status(201).json(user);
});

The validate function is a middleware factory. It takes a schema and returns a middleware function. This pattern lets you reuse the same validation logic across different routes with different schemas. You write validate(userSchema) for user routes and validate(orderSchema) for order routes, and the actual validation logic stays in one place.

Error-handling middleware

Regular middleware has three parameters: req, res, next. Error-handling middleware has four: err, req, res, next. That extra first parameter is how Express knows it is an error handler.

// this goes AFTER all your routes
app.use((err, req, res, next) => {
  console.error(err.stack);
  res.status(err.status || 500).json({
    error: err.message || 'Something went wrong',
  });
});

When any middleware or route calls next(err) with an argument, Express skips everything until it finds a middleware with four parameters. That is your error handler.

app.get('/api/users/:id', async (req, res, next) => {
  try {
    const user = await User.findById(req.params.id);
    if (!user) {
      const err = new Error('User not found');
      err.status = 404;
      return next(err); // goes to the error handler
    }
    res.json(user);
  } catch (err) {
    next(err); // database error, goes to the error handler
  }
});

Without the error handler, unhandled errors crash your server or produce an ugly HTML error page that leaks your stack trace to the client. Neither is what you want.

The error handler must be registered after all other app.use() and route definitions. If you put it at the top, it will never catch anything because Express matches error handlers in order, and errors thrown after the handler's position in the code will not reach it.

A pattern I picked up from a more experienced developer: create custom error classes so your error handler can make smarter decisions.

class AppError extends Error {
  constructor(message, status) {
    super(message);
    this.status = status;
  }
}

// in a route
if (!user) {
  throw new AppError('User not found', 404);
}

// in the error handler
app.use((err, req, res, next) => {
  if (err instanceof AppError) {
    return res.status(err.status).json({ error: err.message });
  }
  // unexpected error, log it, send generic message
  console.error(err);
  res.status(500).json({ error: 'Internal server error' });
});

Known errors get their proper status code and message. Unknown errors get logged for debugging but the client only sees a generic message. You do not want to send a raw database error or a stack trace to whoever is calling your API.

Putting it all together

Here is a stripped-down Express app with middleware in the right order. This is the skeleton most Express projects start from.

const express = require('express');
const morgan = require('morgan');
const app = express();

// 1. Built-in middleware (body parsing)
app.use(express.json());
app.use(express.urlencoded({ extended: true }));

// 2. Third-party middleware (logging)
app.use(morgan('dev'));

// 3. Custom middleware (add request timestamp)
app.use((req, res, next) => {
  req.requestTime = new Date().toISOString();
  next();
});

// 4. Routes
app.get('/', (req, res) => {
  res.json({ status: 'running', time: req.requestTime });
});

app.use('/api/users', require('./routes/users'));
app.use('/api/orders', require('./routes/orders'));

// 5. 404 handler (no route matched)
app.use((req, res) => {
  res.status(404).json({ error: 'Route not found' });
});

// 6. Error handler (must be last, must have 4 parameters)
app.use((err, req, res, next) => {
  console.error(err.stack);
  res.status(err.status || 500).json({
    error: err.message || 'Internal server error',
  });
});

app.listen(3000, () => console.log('running on 3000'));

The order follows a pattern: parsing goes first so req.body is available everywhere, logging goes next so every request gets recorded, custom middleware goes after that, then routes, then the 404 catch-all for unmatched routes, and the error handler goes dead last.

The 404 handler is a regular middleware with no path argument. Since it is registered after all routes, Express only reaches it when no route matched the incoming request. It does not have four parameters, so it is not an error handler. It just sends a 404 response.

Common mistakes I made (so you do not have to)

Forgetting next(): The request hangs and the client eventually times out. No error in the console. If a route is mysteriously not responding, check if a middleware above it forgot to call next().

Calling next() after sending a response: Causes the "headers already sent" error. Use return before res.send() or res.json() in conditional blocks.

Putting middleware after routes: The middleware never runs for those routes because Express already matched and handled the request.

Not using express.json(): req.body is undefined and you blame your frontend, your database, your HTTP client, and possibly the universe before realizing you forgot one line.

Writing async middleware without try/catch: Unhandled promise rejections in Express 4 do not trigger the error handler. They crash the process or get silently swallowed. Wrap async middleware in try/catch and pass errors to next(err). Express 5 fixes this, but most projects are still on 4.

// Express 4: you need this wrapper
app.get('/api/data', async (req, res, next) => {
  try {
    const data = await fetchSomething();
    res.json(data);
  } catch (err) {
    next(err);
  }
});

What to read next

Middleware is the pattern that holds an Express app together. Once you understand the request pipeline and the role of next(), most Express features make more sense. The concepts you will run into next build directly on this: route grouping with express.Router(), authentication strategies with Passport.js, more sophisticated validation with Zod or Joi, rate limiting with express-rate-limit, and CORS handling with the cors package. All of them are middleware. They follow the same (req, res, next) pattern you just learned.

If something in your Express app is not working and you cannot figure out why, add a console.log to each middleware and check the order. Nine times out of ten, the answer is either a missing next(), a misplaced app.use(), or middleware running in the wrong sequence.

References

• Express documentation: Using middleware

• Express documentation: Writing middleware

• Express documentation: Error handling

• Express documentation: express.json()

• Express documentation: express.Router()

• morgan: HTTP request logger middleware

• Joi: schema-based validation

• Zod: TypeScript-first schema validation

• jsonwebtoken: JWT implementation for Node.js

• express-rate-limit: rate limiting middleware

I remember the first time someone told me I could write server-side code in JavaScript. My reaction was something like "wait, that does not sound right." JavaScript was the language that ran inside Chrome. How would it read files from a hard drive? How would it listen for HTTP requests? It did not have access to any of that. The browser would never let it.

They were talking about Node.js.

Why JavaScript could not run outside the browser

This confused me until I understood what JavaScript actually is versus what runs it.

JavaScript the language is just a specification. It defines syntax, data types, how loops work, how functions behave. It does not say anything about the DOM, or window, or document.getElementById. Those are APIs that the browser provides. The browser says "here, JavaScript, I will give you access to this object called document and you can use it to manipulate the page." JavaScript itself has no idea what a webpage is.

The thing that executes JavaScript code is called a runtime. In a browser, the runtime is the browser itself, specifically a JavaScript engine embedded inside it. Chrome uses an engine called V8. Firefox uses SpiderMonkey. Safari uses JavaScriptCore. Each one reads your JavaScript, compiles it into machine code, and runs it. But they also provide browser-specific APIs like fetch, localStorage, and setTimeout that let your code interact with the browser environment.

So JavaScript was not really trapped in the browser. The engines that ran it were trapped in the browser. The language itself was perfectly general. Someone just had to take one of those engines out of the browser and give it a different set of APIs.

That someone was Ryan Dahl, and the thing he built in 2009 was Node.js.

What Node.js actually is

Node.js took Chrome's V8 engine, pulled it out of the browser, and wrapped it with APIs for things that servers need. File system access. Network sockets. HTTP request handling. Process management. Child processes. Streams. Buffers for raw binary data. All the things the browser would never let JavaScript touch.

So when you write JavaScript in Node.js, the core language is identical. Variables, functions, arrays, objects, promises, async/await, all the same. What changes is what is available around the language. There is no document. There is no window. There is no DOM. Instead you get fs for reading and writing files, http for building web servers, path for working with file paths, and process for interacting with the operating system.

// This runs in Node.js, not in a browser
const fs = require('fs');
const http = require('http');

// read a file from the hard drive
const data = fs.readFileSync('config.json', 'utf-8');
console.log(data);

// start an HTTP server that listens on port 3000
const server = http.createServer((req, res) => {
  res.writeHead(200, { 'Content-Type': 'text/plain' });
  res.end('Hello from Node.js');
});

server.listen(3000, () => {
  console.log('Server running at http://localhost:3000');
});

Try running fs.readFileSync in a browser console. You get an error. The browser has no idea what fs is. Try running document.querySelector in Node.js. Same thing, other direction. Same language, different environment.

// browser console
> fs.readFileSync('test.txt')
  Uncaught ReferenceError: fs is not defined

// node repl
> document.querySelector('div')
  ReferenceError: document is not defined

V8 under the hood

You do not need to know how V8 works internally to use Node.js. But knowing the basics helps when people throw around terms like "JIT compilation" or "single-threaded" and expect you to nod along.

V8 compiles JavaScript to machine code. Older JavaScript engines used to interpret code line by line, which was slow. V8 compiles it first, which makes execution much faster. It also uses something called Just-In-Time (JIT) compilation, where it watches which parts of your code run frequently and optimizes those sections further while the program is already running.

The part that matters practically: V8 is fast. Not fast for a scripting language, just fast. This is a big reason Node.js took off. Before V8, running JavaScript outside the browser would have been painfully slow for server workloads. V8 made it competitive with languages that had been doing backend work for decades.

The event loop and why Node.js handles requests differently

This is the concept that made Node.js click for me, and also the one that confused me the longest.

Traditional backend runtimes like PHP or Java (with thread-per-request models) handle each incoming request by assigning it a thread. A thread is like a worker. One request comes in, one worker picks it up, does everything the request needs (database query, file read, external API call), and sends the response. If 1000 requests come in at once, you need close to 1000 threads. Each thread uses memory. Each one sits around waiting when it hits a database query that takes 200ms to come back. A lot of resources spent on waiting.

Node.js uses a single thread with an event loop. One worker handles all incoming requests. When that worker hits something slow, like a database query, it does not wait. It says "let me know when the result comes back" and moves on to handle the next request. When the database responds, a callback fires and the worker picks up where it left off.

// request comes in -> wait for database -> wait -> wait -> respond

// Node does this:
// request comes in -> ask database -> move to next request
// database responds -> pick up first request -> respond

const http = require('http');

const server = http.createServer((req, res) => {
  // simulate a database call that takes 2 seconds
  setTimeout(() => {
    res.end('Data from database');
  }, 2000);
  // Node does NOT freeze here for 2 seconds
  // it goes and handles other requests while waiting
});

server.listen(3000);

The tradeoff is real though. If you write code that does heavy computation directly on that single thread, like crunching numbers in a loop for 5 seconds, nothing else runs during that time. Every other request waits. Node.js is excellent at I/O-heavy work (database queries, API calls, file reads) where the program spends most of its time waiting for external things. It is not the best choice for CPU-heavy work like image processing or complex math, unless you offload that to worker threads or separate services.

How Node.js compares to PHP and Java

When I was deciding what to learn for backend, I kept reading comparisons that were more tribal allegiance than actual explanation. Here is what actually differs.

PHP traditionally runs one process per request. A request comes in, PHP starts up, runs the script, sends the response, and the process dies. The next request starts fresh. This is simple and means one request cannot easily corrupt another, but it also means there is overhead in starting up for every single request. Newer PHP (with tools like Swoole or FrankenPHP) can keep processes alive, but the traditional model is request-and-die.

Java with something like Spring Boot runs a multi-threaded server. Each request gets a thread from a thread pool. This handles concurrency well but threads have memory overhead, and context switching between many threads has a cost. Java is fast and battle-tested for large enterprise systems, but it takes more boilerplate code to get a simple server running.

Node.js sits in a different spot. One thread, non-blocking I/O, the event loop. Less memory per connection than a thread-per-request model. The cost is that your code has to be written in an asynchronous style, and CPU-heavy tasks need special handling.

The reason a lot of JavaScript developers picked Node.js over learning PHP or Java is straightforward: they already knew the language. One language for the frontend and the backend. One set of syntax rules to remember. Shared code between browser and server when it made sense. Shared tooling (npm, ESLint, Prettier). For small teams and startups building web apps, that was a real productivity win.

Where Node.js gets used in practice

Node.js runs the backend for a lot of companies you have probably used. Netflix moved parts of their stack to Node.js and reported faster startup times. LinkedIn rebuilt their mobile backend in Node.js and reduced their server count. PayPal rewrote parts of their Java backend in Node.js and saw pages render faster.

But the practical everyday uses are more grounded than big company case studies. These are the places where Node.js shows up most often.

REST APIs and GraphQL servers. If you are building a web or mobile app that needs a backend to handle data, Node.js with Express or Fastify is one of the most common setups. Lightweight, quick to build, handles many concurrent connections well.

Real-time applications. Chat apps, live notifications, collaborative editors, multiplayer game servers. The event loop and WebSocket support make Node.js a natural fit for anything where the server needs to push data to clients without them asking for it.

Command-line tools. npm itself is written in Node.js. So are tools like ESLint, Prettier, Webpack, and Vite. If you have installed anything with npm install -g, you have been running Node.js programs on your machine.

Build tooling and automation. Task runners, bundlers, dev servers, testing frameworks. The JavaScript ecosystem's entire build pipeline runs on Node.js.

Server-side rendering. Frameworks like Next.js run on Node.js to render React pages on the server before sending them to the browser. This is how a lot of modern websites ship fast initial page loads.

// A simple REST API endpoint using Express
const express = require('express');
const app = express();

app.use(express.json());

const users = [];

app.get('/users', (req, res) => {
  res.json(users);
});

app.post('/users', (req, res) => {
  const user = { id: users.length + 1, name: req.body.name };
  users.push(user);
  res.status(201).json(user);
});

app.listen(3000, () => {
  console.log('API running on http://localhost:3000');
});

That is a working API. Fourteen lines of actual logic. In Java with Spring Boot, the equivalent setup involves annotations, dependency injection configuration, a controller class, and a build file. Both approaches are valid, but one of them gets you to a working prototype in two minutes.

What to take away from this

Node.js is V8 plus system-level APIs, packaged as a runtime that lets JavaScript run outside the browser. The language is the same. The environment is different. The event loop makes it handle concurrent I/O well, but CPU-heavy work needs a different approach.

You do not need to understand V8 internals or event loop phases to start building things. Write a small Express server. Hit it with a few requests. Read files from disk. Connect to a database. The concepts land faster when you have running code in front of you than when you are reading about them.

In the next post we will set up a Node.js project from scratch and dig into how require, module.exports, and the module system actually work. That is where the real building starts.

References

• Node.js official documentation - start with the "Getting Started" guides

• V8 JavaScript engine - Chrome's JS engine that powers Node.js

• MDN: JavaScript runtime - good explanation of engines vs runtimes

• Node.js event loop explained - the official guide to how the event loop works

• Express.js - the most popular Node.js web framework for beginners

• Fireship: Node.js in 100 Seconds - quick visual overview, worth watching before going further

I had this wrong mental model for weeks. I kept picturing a single thread like a single person doing everything, start to finish, one task at a time. That is not what happens. And once I understood what actually happens, a lot of things about Node clicked that had confused me before.

This post is about that. How Node.js works under the hood, why one thread is not as limiting as it sounds, and where it actually falls apart.

Threads and processes, without the textbook definition

A process is a running program. When you open VS Code, that is a process. When you start a Node server with node server.js, that is a process too. Each process gets its own chunk of memory from the operating system and runs independently.

A thread lives inside a process. It is a single sequence of instructions being executed. A process can have many threads running at the same time, sharing the same memory. That is how languages like Java handle multiple users: spin up a new thread for each incoming request, let them all run in parallel.

Node.js does not do that. Your JavaScript code runs on one thread. One. If you write a function that takes 3 seconds to finish, nothing else runs during those 3 seconds. No other request gets processed, no other callback fires. The whole server just sits there waiting.

That sounds terrible. But Node gets away with it, and the reason is the event loop.

The chef analogy

Think of a restaurant kitchen with one chef. This chef cannot clone themselves. There is only one.

A regular multi-threaded server is like hiring a new chef for every order. Ten orders come in, ten chefs are cooking simultaneously. Works fine until you have 10,000 orders and need 10,000 chefs and your kitchen is on fire.

Node's approach is different. The one chef takes an order, puts the pasta on the stove, and immediately moves to the next order. They do not stand at the stove watching water boil. They chop vegetables for order two, put that in the oven, then check if the pasta is done. When the oven timer goes off, they plate that dish. One chef, many dishes in progress, none of them blocking the others.

The chef is your single JavaScript thread. The stove, the oven, the timers are the operating system and the thread pool handling the slow stuff in the background. The thing connecting it all, the system that tells the chef "the pasta is ready" or "the oven timer went off," that is the event loop.

One chef, many dishes in progress:

  Order 1: pasta on stove       [waiting... ready!] --> plate it
  Order 2: veggies chopping --> oven [waiting....... ready!] --> plate it
  Order 3: salad prep --> done immediately --> plate it
  Order 4: soup heating        [waiting.... ready!] --> plate it

  The chef moves between tasks. They never just stand and wait.

What the event loop actually does

When your Node server receives a request, your JavaScript code starts running. If that code does something synchronous like math, string manipulation, or building a JSON response, it runs right there on the main thread and finishes.

But most server work is not like that. Reading a file from disk, querying a database, making an HTTP call to another API. These are I/O operations, and they are slow compared to CPU work. Disk reads take milliseconds. Network calls take tens or hundreds of milliseconds. If the main thread sat around waiting for each of these, the server would be useless.

So Node does not wait. When your code says "read this file," Node hands that job to the operating system. The OS does the actual reading on its own, separate from your JavaScript thread. Node registers a callback function, basically a note saying "when the file is done, run this function." Then it moves on immediately to handle the next thing.

The event loop is the mechanism that checks whether any of those background tasks have finished. It runs in a continuous cycle. Each cycle, it looks at a queue of completed tasks, picks up the associated callbacks, and runs them one at a time on the main thread.

const fs = require('fs');

console.log('before reading file');

fs.readFile('/some/big/file.txt', 'utf8', (err, data) => {
  // this callback runs later, when the file is done being read
  console.log('file has been read');
});

console.log('after calling readFile');

Output:

before reading file
after calling readFile
file has been read

The readFile call returns immediately. It does not block. The callback runs later, after the event loop picks it up from the completed-tasks queue. Between calling readFile and the callback firing, Node is free to handle other requests.

The background workers

"But someone has to actually read the file." Yes. Node is not doing magic. The work still happens, just not on your JavaScript thread.

Node.js is built on top of a C library called libuv. libuv maintains a thread pool, which defaults to 4 threads. When you call fs.readFile, libuv assigns the actual disk read to one of those pool threads. When that thread finishes, it places the result in a queue. The event loop picks it up on the next cycle and runs your callback.

For network I/O, it is even more efficient. libuv uses operating system features like epoll on Linux or kqueue on macOS. These let a single thread monitor thousands of network sockets without dedicating a thread to each one. That is why Node handles network-heavy workloads particularly well.

Your code (main thread)          libuv (background)
--------------------------       ---------------------------
fs.readFile('data.json')  --->   Worker thread 1: reading file
db.query('SELECT ...')    --->   OS network I/O: waiting for DB
fetch('https://api...')   --->   OS network I/O: waiting for response
                                 
  Main thread is free to         Each finishes independently
  handle new requests             and queues the result
                                 
  Event loop picks up results <--- callback queue
  and runs your callbacks

You can increase the thread pool size if your app does a lot of file system work:

// set this before any I/O happens, usually at the very top of your entry file
process.env.UV_THREADPOOL_SIZE = 8;

The default of 4 is fine for most apps. If you are doing heavy file processing or DNS lookups (which also use the pool), bumping it up can help.

Handling multiple clients

Here is a concrete example. Say your Node server gets three requests at roughly the same time. Each one queries a database, which takes about 50ms.

const express = require('express');
const app = express();

app.get('/user/:id', async (req, res) => {
  const user = await db.findUser(req.params.id);  // ~50ms, non-blocking
  res.json(user);
});

app.listen(3000);

What happens:

1. Request A arrives. Node starts running the handler, hits await db.findUser(), sends the query to the database over the network, and moves on. The main thread is free.

2. Request B arrives a few milliseconds later. Same thing. Query sent, main thread free.

3. Request C arrives. Same deal.

4. The database responds to request A's query. The event loop picks up the callback, Node runs res.json(user) and sends the response.

5. Request B's query finishes. Same process.

6. Request C's query finishes.

All three requests were "in progress" at the same time, even though only one thread ran JavaScript. The actual waiting happened elsewhere. The main thread only did the quick parts: parsing the request, building the response, sending it out.

This is concurrency without parallelism. Multiple things are in progress at the same time, but only one piece of JavaScript runs at any given instant. The parallelism happens in the background, in libuv's thread pool and the OS kernel's network handling.

Where this breaks down

The event loop works beautifully for I/O. It falls apart when you put heavy computation on the main thread.

app.get('/heavy', (req, res) => {
  // this blocks the entire server
  let sum = 0;
  for (let i = 0; i < 1_000_000_000; i++) {
    sum += i;
  }
  res.json({ sum });
});

While that loop runs, every other request to your server sits in a queue, waiting. No callbacks fire. No responses go out. The event loop cannot cycle because your code is hogging the thread. If that loop takes 2 seconds, every user experiences a 2-second delay, not just the one who hit /heavy.

This is the tradeoff. Node gives you a fast, lightweight concurrency model for I/O-bound work. In exchange, you have to keep the main thread clear of heavy computation.

When you do need CPU-intensive work, Node gives you worker_threads:

const { Worker } = require('worker_threads');

app.get('/heavy', (req, res) => {
  const worker = new Worker('./heavy-calc.js');
  
  worker.on('message', (result) => {
    res.json({ sum: result });
  });
  
  worker.on('error', (err) => {
    res.status(500).json({ error: err.message });
  });
});

// heavy-calc.js
const { parentPort } = require('worker_threads');

let sum = 0;
for (let i = 0; i < 1_000_000_000; i++) {
  sum += i;
}

parentPort.postMessage(sum);

The heavy calculation runs on a separate thread. The main thread stays free. The worker sends the result back when it is done. Your other routes keep responding normally while the calculation runs.

Why Node scales well for the right workload

Traditional multi-threaded servers allocate a thread per connection. Each thread consumes memory (typically around 1-2MB for the stack alone). Ten thousand concurrent connections means ten thousand threads and gigabytes of memory just for thread stacks, before your application code does anything.

Node uses one thread for JavaScript and lets the OS handle the waiting. A single Node process can hold tens of thousands of concurrent connections because most of those connections are just sitting in an OS-level queue, waiting for data. The memory cost per connection is tiny.

This is why Node became popular for real-time applications, chat servers, API gateways, streaming services. These are all I/O-heavy, CPU-light workloads where thousands of connections are open but most of them are idle at any given moment.

It is not the right tool for everything. Image processing, video encoding, machine learning inference, anything that pins the CPU for extended periods is a poor fit for Node's main thread. You can work around it with worker threads or by calling out to separate services, but at that point you are fighting the architecture instead of working with it.

A quick summary of the mental model

Your JavaScript runs on one thread. When it hits an I/O operation (file read, database query, network call), it hands that off to the system and keeps going. The event loop continuously checks whether any of those operations finished. When one finishes, the event loop runs the associated callback on the main thread.

The golden rule: never block the event loop. Keep the main thread doing quick work. Push slow I/O to the system. Push heavy computation to worker threads. If you follow that, one thread handles more concurrent users than you would expect.

// good: non-blocking I/O
const data = await fs.promises.readFile('data.json', 'utf8');

// bad: blocking the event loop
const data = fs.readFileSync('data.json', 'utf8');

The readFileSync version freezes your server until the file is read. The await version lets the event loop keep cycling. In a script you run once, readFileSync is fine. In a server handling requests, it is not.

References

• Node.js docs: The event loop - the official explanation, worth reading after this post

• Node.js docs: Worker threads - for CPU-bound work

• libuv documentation - the C library underneath Node that handles the thread pool and OS-level I/O

• Philip Roberts: What the heck is the event loop anyway? - the classic JSConf talk, still the best visual explanation out there

• Node.js docs: Don't block the event loop - practical advice on keeping your server responsive

This post covers what URL parameters and query strings are, how they differ, and how you actually use them in Express. If you have been copy-pasting route code without fully understanding the :id part or the req.query part, this should clear that up.

A URL has parts, and they have names

Before anything else, look at a URL and know what each piece is called. Take this one:

https://api.foodapp.com/restaurants/5f2b/menu?category=biryani&sort=price

The path is /restaurants/5f2b/menu. The 5f2b in there is a URL parameter, a variable piece embedded in the path itself. Everything after the ? is the query string. category=biryani and sort=price are query parameters, key-value pairs separated by &.

Two different mechanisms. Same URL. Different jobs.

URL parameters: the "which one" part

URL parameters identify a specific resource. When you write /users/42, you are saying "I want user number 42." Not a list of users. Not a filtered set. One specific user.

In Express, you define a URL parameter by putting a colon before the name in your route:

// :id is the parameter
app.get('/users/:id', (req, res) => {
  console.log(req.params);
  // { id: '42' }

  console.log(req.params.id);
  // '42'
});

When someone hits GET /users/42, Express matches the route and puts 42 into req.params.id. The name after the colon is what you use to access it. Call it :id, :userId, :slug, whatever makes sense for your route.

You can have multiple parameters in one route. A food delivery app might need both a restaurant and a menu item:

app.get('/restaurants/:restaurantId/menu/:itemId', (req, res) => {
  console.log(req.params);
  // { restaurantId: '5f2b', itemId: 'a1c3' }
  
  const restaurant = await Restaurant.findById(req.params.restaurantId);
  const item = restaurant.menu.id(req.params.itemId);
  
  res.json(item);
});

The URL /restaurants/5f2b/menu/a1c3 fills both params. Each :name in the route definition maps to the matching segment in the actual URL, left to right.

One thing to watch: req.params.id is always a string. Even if the URL is /users/42, you get the string '42', not the number 42. If you are passing it to a database query that expects a number, you need to convert it yourself. MongoDB ObjectIds work fine as strings, but if your IDs are numeric you will want parseInt(req.params.id) or Number(req.params.id).

Query strings: the "how do you want it" part

Query strings modify a request. They filter, sort, paginate, or adjust what comes back, but they do not change which resource you are looking at. The URL /users means "all users." The URL /users?role=rider&sort=name still means "all users," just filtered to riders and sorted by name.

Express parses query strings automatically. Everything after the ? ends up in req.query:

// URL: /users?role=rider&sort=name&page=2
app.get('/users', (req, res) => {
  console.log(req.query);
  // { role: 'rider', sort: 'name', page: '2' }

  console.log(req.query.role);
  // 'rider'

  console.log(req.query.page);
  // '2' (still a string)
});

Same as params, everything in req.query is a string. page comes back as '2', not 2. Parse it before doing math with it.

Query parameters are optional by nature. If someone hits /users without any query string, req.query is just an empty object {}. Your code needs to handle that. Default values are a good habit:

app.get('/users', async (req, res) => {
  const page = parseInt(req.query.page) || 1;
  const limit = parseInt(req.query.limit) || 10;
  const sort = req.query.sort || 'createdAt';
  const role = req.query.role; // undefined if not provided

  const filter = {};
  if (role) filter.role = role;

  const users = await User.find(filter)
    .sort(sort)
    .skip((page - 1) * limit)
    .limit(limit);

  res.json(users);
});

This route works whether you call it as /users, /users?page=3, or /users?role=admin&sort=name&page=2&limit=5. The defaults cover whatever the caller did not specify.

The difference, shown side by side

The short version: params say "which thing," query strings say "how you want it."

If you are fetching one specific user, that is a param: /users/42. If you are searching for users who match some criteria, those criteria go in the query string: /users?city=mumbai&role=rider. If you are getting a specific order for a specific user, both user and order are params: /users/42/orders/99. If you want that order's invoice in PDF format instead of JSON, that is a query: /users/42/orders/99?format=pdf.

I used to overthink this. The test I use now is simple: would the route still make sense without this piece of data? If removing it makes the URL meaningless (like /users/ with no ID when you want one user), it is a param. If removing it just gives you a broader or default response, it is a query.

Using both at the same time

Most real routes use params and query strings together. Here is a route that fetches orders for a specific user, with optional filtering:

// GET /users/:userId/orders?status=delivered&page=1
app.get('/users/:userId/orders', async (req, res) => {
  const { userId } = req.params;
  const { status, page = 1, limit = 10 } = req.query;

  const filter = { user: userId };
  if (status) filter.status = status;

  const orders = await Order.find(filter)
    .sort({ createdAt: -1 })
    .skip((parseInt(page) - 1) * parseInt(limit))
    .limit(parseInt(limit))
    .populate('restaurant', 'name');

  res.json(orders);
});

The param :userId tells us whose orders we are looking at. The query strings status, page, and limit let the caller control the response shape. Remove the query strings and you still get a valid response: all orders for that user, first page, default limit. Remove the param and the route does not match at all.

A few things that tripped me up

Params are positional. Express matches them left to right in the URL path. If you have two routes like app.get('/users/:id') and app.get('/users/active'), the order you define them matters. Express checks routes in the order they are registered. If :id comes first, a request to /users/active matches it with req.params.id equal to 'active'. Put the specific route before the parameterized one:

// specific route first
app.get('/users/active', (req, res) => {
  // handles /users/active
});

// parameterized route second
app.get('/users/:id', (req, res) => {
  // handles /users/42, /users/abc, etc.
});

Query strings do not affect route matching. Express matches routes based on the path only. /users and /users?role=admin hit the same route handler. You never define query parameters in your route string.

Empty query values are valid. A URL like /users?name= gives you req.query.name as an empty string '', which is truthy in a conditional check... actually no, empty strings are falsy in JavaScript. So if (req.query.name) fails on both undefined and ''. If you need to distinguish between "not provided" and "provided but empty," check for undefined explicitly:

if (req.query.name !== undefined) {
  // they sent the parameter, even if it is empty
}

When to use which

Params for identification: /products/:productId, /orders/:orderId, /users/:userId/profile.

Query strings for everything optional: search terms, filters, sort order, pagination, format preferences, feature flags.

If you catch yourself putting optional data in the path, move it to the query string. If you catch yourself using query strings to identify a specific resource, move it to a param. The URL should read like a sentence. /restaurants/5f2b/menu reads as "the menu for restaurant 5f2b." That makes sense. /restaurants?id=5f2b&show=menu technically works but reads like a database query leaked into your URL.

REST API conventions reinforce this pattern. You will see it everywhere once you start reading API documentation for services like Stripe, GitHub, or Twilio. The resource goes in the path. The options go after the question mark.

References

• Express routing guide

• Express req.params docs

• Express req.query docs

• MDN: What is a URL

• REST API design guidelines on URL structure

So I went and figured it out. The mental model that finally made it click for me was a restaurant, which I will get to in a second. But first, the thing that confused me most.

Node.js runs on one thread. That sounds terrible.

Most backend technologies like Java or Python spin up a new thread for every incoming request. Thread gets a request, does the work, sends the response, dies. Simple. If you have 1,000 requests coming in at once, you have 1,000 threads running. Each thread takes up memory and CPU time. At some point the server runs out of threads and starts queuing, or worse, crashing.

Node.js does not do this. It runs your JavaScript code on a single thread. One. That is it. When I first read that, my reaction was: how does that not immediately fall over?

The answer is in what Node does with that single thread. It never lets it sit around waiting.

The restaurant analogy

Think of a traditional multi-threaded server as a restaurant where every table gets a dedicated waiter. The waiter takes the order, walks it to the kitchen, stands at the kitchen window waiting for the food, then walks it back to the table. While the waiter is standing at the kitchen window doing nothing, they cannot help anyone else. If ten tables need service at the same time, you need ten waiters standing around staring at kitchen windows.

Node.js is the restaurant with one waiter who is extremely good at their job. The waiter takes the order from table one, hands the ticket to the kitchen, and immediately walks to table two. Takes that order, hands it to the kitchen, checks on table three. When the kitchen rings the bell saying table one's food is ready, the waiter picks it up and delivers it. The waiter never stands at the kitchen window. They are always moving, always taking the next task that is ready.

The kitchen in this analogy is the operating system doing I/O: reading files from disk, making network requests, querying databases. That work takes time, but your JavaScript thread does not need to be the one waiting for it. The OS handles it in the background, and when the result comes back, Node puts a callback in a queue. The thread picks it up when it is free.

That is non-blocking I/O. Your code says "go read this file" and moves on. It does not pause. It does not wait. It handles the next thing.

const fs = require('fs');

// non-blocking: Node hands this to the OS and moves on
fs.readFile('/some/big/file.txt', 'utf8', (err, data) => {
  // this runs later, when the file is actually read
  console.log('file contents loaded');
});

// this runs immediately, before the file is done reading
console.log('I did not wait for the file');

Output:

I did not wait for the file
file contents loaded

The second console.log runs first because readFile does not block execution. The callback fires later, when the OS finishes reading the file. If you have used fetch in the browser, you already understand this pattern. Same idea, server side.

Blocking vs non-blocking: what the difference looks like in code

Here is the blocking version of that same file read:

const fs = require('fs');

// blocking: the thread stops here until the file is fully read
const data = fs.readFileSync('/some/big/file.txt', 'utf8');
console.log('file contents loaded');

// this cannot run until the file read is completely done
console.log('now I can do other things');

Output:

file contents loaded
now I can do other things

If that file takes 500ms to read, the entire server is frozen for 500ms. No other request gets handled. Nobody gets a response. The thread is parked at the kitchen window.

In a real app with real traffic, that 500ms freeze means every user currently waiting for a response is also waiting for your file read to finish. This is why Node's standard library has async versions of almost every function. The Sync versions exist for startup scripts and CLI tools where blocking is fine because there is no one else waiting.

The event loop

The event loop is the mechanism that makes all of this work. It is a loop that runs continuously, checking: is there a callback ready to execute? If yes, run it. If no, wait for one.

   ┌───────────────────────────┐
┌─>│         timers            │  setTimeout, setInterval callbacks
│  └─────────────┬─────────────┘
│  ┌─────────────┴─────────────┐
│  │     pending callbacks     │  I/O callbacks deferred to next loop
│  └─────────────┬─────────────┘
│  ┌─────────────┴─────────────┐
│  │       idle, prepare       │  internal use only
│  └─────────────┬─────────────┘
│  ┌─────────────┴─────────────┐
│  │         poll              │  retrieve new I/O events
│  └─────────────┬─────────────┘
│  ┌─────────────┴─────────────┐
│  │         check             │  setImmediate callbacks
│  └─────────────┬─────────────┘
│  ┌─────────────┴─────────────┐
│  │    close callbacks        │  socket.on('close', ...)
│  └─────────────┬─────────────┘
└─────────────────┘

You do not need to memorize these phases. What matters is the mental model: your code runs, hands off slow work to the OS, and the event loop picks up the results when they are ready. Everything async in Node, from reading files to database queries to HTTP requests, goes through this cycle.

Here is a quick way to see the order for yourself:

console.log('1: synchronous, runs first');

setTimeout(() => {
  console.log('4: timer callback, runs in the timers phase');
}, 0);

setImmediate(() => {
  console.log('5: setImmediate, runs in the check phase');
});

process.nextTick(() => {
  console.log('2: nextTick, runs before any phase');
});

Promise.resolve().then(() => {
  console.log('3: promise, runs after nextTick but before timers');
});

Output:

1: synchronous, runs first
2: nextTick, runs before any phase
3: promise, runs after nextTick but before timers
4: timer callback, runs in the timers phase
5: setImmediate, runs in the check phase

process.nextTick runs before the event loop continues to the next phase. Promises run right after that. setTimeout and setImmediate come later. This ordering matters when you are debugging race conditions or figuring out why a callback fires before another. Run this snippet, stare at the output, modify it. That teaches the order faster than reading about it.

Concurrency is not parallelism

This is where people get confused. Node handles many requests concurrently. It does not handle them in parallel. The difference matters.

Concurrency means managing multiple things at once. Parallelism means doing multiple things at the same time. Node is concurrent. It juggles thousands of requests by never waiting for any single one to finish. But all your JavaScript runs on one thread, so two pieces of JS code never execute at the exact same moment.

For I/O-heavy work (reading files, calling APIs, querying databases), this is completely fine. The thread spends almost no time on each request because it hands off the slow part and moves on. A single Node process can handle tens of thousands of concurrent connections this way.

For CPU-heavy work (image processing, video encoding, heavy math), this falls apart. If your code runs a loop that takes 3 seconds of pure computation, nothing else happens during those 3 seconds. The event loop is blocked. Every request in the queue sits there waiting.

// this blocks the event loop for ~3 seconds
// no request gets handled during this time
function heavyComputation() {
  let sum = 0;
  for (let i = 0; i < 3_000_000_000; i++) {
    sum += i;
  }
  return sum;
}

// while heavyComputation runs, the server is frozen
app.get('/slow', (req, res) => {
  const result = heavyComputation();
  res.json({ result });
});

// this route also stops responding while /slow is computing
app.get('/fast', (req, res) => {
  res.json({ message: 'hello' });
});

Hit /slow once and try hitting /fast in another tab. It hangs until the computation finishes. One bad route takes down the entire server.

If you need to do CPU-heavy work in Node, the answer is worker_threads. They run JavaScript in a separate thread so the main event loop stays free:

const { Worker } = require('worker_threads');

app.get('/slow', (req, res) => {
  const worker = new Worker('./heavy-computation.js');
  worker.on('message', (result) => {
    res.json({ result });
  });
  worker.on('error', (err) => {
    res.status(500).json({ error: err.message });
  });
});

The computation happens off the main thread. The event loop keeps handling other requests. Your /fast route stays responsive.

Where Node is a good fit and where it is not

Node works well for anything that is mostly I/O with a thin layer of logic on top. REST APIs, GraphQL servers, real-time apps with WebSockets, chat applications, streaming services, proxy servers. The pattern is the same: receive a request, ask some other service or database for data, send the response. The thread barely does any work itself. It is a traffic controller.

Node is a poor fit for raw computation. If your app resizes images on every request, transcodes video, runs machine learning inference, or does heavy data crunching, a single-threaded runtime is working against you. You can use worker threads, but at that point you might be better off with Go, Rust, or Python with proper multiprocessing depending on the workload.

The honest answer is that most web applications are I/O-bound. Most of the time your server is waiting: waiting for the database, waiting for an external API, waiting for a file to be read. Node was built exactly for that waiting problem.

Who actually uses Node.js in production

Netflix runs parts of their backend on Node. They switched from Java for some services and saw startup times drop significantly. Their UI layer runs on Node because server-side rendering with React was simpler there than in a Java stack.

PayPal rewrote their account overview page from Java to Node. Their team reported being able to build the Node version with fewer people in less time, and the resulting app handled more requests per second with lower average response times.

LinkedIn moved their mobile backend from Ruby on Rails to Node. They went from running 30 servers to 3, partly because Node handled the same concurrent load with fewer resources.

Uber's matching system, the part that pairs riders with drivers, runs on Node. The system needs to handle massive numbers of concurrent connections with low latency, which is exactly the kind of I/O-bound real-time workload Node handles well.

These are not small experiments. They are production systems handling millions of users. The pattern across all of them is the same: lots of concurrent connections, mostly I/O, relatively light computation per request.

Wrapping up

Node is fast because it never lets its single thread sit idle. Instead of dedicating a thread to each request and letting it wait around, Node hands off the slow work and moves on to the next thing. The event loop picks up the results when they are ready. That model lets a single process handle thousands of concurrent connections without the memory overhead of thousands of threads.

The tradeoff is CPU-bound work. Block the event loop and everything stops. Know when you are doing I/O (use async, let Node do its thing) and when you are doing computation (use worker threads or pick a different tool).

If you are building APIs, real-time features, or anything that talks to databases and external services, Node is a strong choice. Understand the event loop, keep it free, and it will handle more traffic than you expect from a single process.

References

• Node.js docs: The event loop - the official explanation with all the phases

• Node.js docs: Worker threads - how to offload CPU work

• Node.js docs: File system - async vs sync file operations

• Philip Roberts: What the heck is the event loop anyway? - the best visual explanation of the event loop, 26 minutes that will save you hours

• Node.js at Netflix - their experience migrating from Java

• PayPal engineering blog on Node.js - performance comparisons with their Java stack

• libuv documentation - the C library that handles Node's async I/O under the hood

That was the moment I realized file uploads are a different kind of HTTP request, and your server needs help handling them.

Why your server cannot read files from a normal form

When you submit a regular HTML form with text fields, the browser encodes the data as application/x-www-form-urlencoded. That is the default. It looks like a query string: name=Priya&email=priya@example.com. Express's built-in express.json() and express.urlencoded() middleware can parse this without any trouble.

Files do not work that way. A file is binary data. It could be an image, a PDF, a video. You cannot flatten that into a query string. So the browser switches to a different encoding called multipart/form-data. The request body gets split into multiple parts, each separated by a randomly generated boundary string. One part might be a text field, another part is the raw file bytes with metadata like the original filename and MIME type.

Express has no built-in parser for multipart data. It sees the request come in, does not recognize the encoding, and skips it. That is why req.body was empty. The data was there in the request. Express just did not know how to read it.

<!-- this form sends multipart/form-data -->
<form action="/upload" method="POST" enctype="multipart/form-data">
  <input type="text" name="caption" />
  <input type="file" name="photo" />
  <button type="submit">Upload</button>
</form>

Without enctype="multipart/form-data", the browser sends the filename as a string but not the actual file contents. I made this mistake twice before it stuck.

What Multer does

Multer is a Node.js middleware built specifically for handling multipart/form-data. It parses the incoming request, extracts the file(s), saves them wherever you tell it to, and attaches file metadata to req.file or req.files so your route handler can use it.

You install it like any other npm package:

npm install multer

Multer only processes multipart forms. If someone sends a regular JSON request to an endpoint that uses Multer, it ignores the request and passes it along unchanged. It does not interfere with your other routes.

Handling a single file upload

The simplest case: one file input, one upload. Multer gives you a function called upload.single('fieldname') that handles this. The field name must match the name attribute on your HTML file input.

const express = require('express');
const multer = require('multer');

const app = express();
const upload = multer({ dest: 'uploads/' });

app.post('/upload', upload.single('photo'), (req, res) => {
  console.log(req.file);
  console.log(req.body); // text fields still work
  res.json({ file: req.file });
});

app.listen(3000);

When you submit the form, Multer saves the file to the uploads/ folder and populates req.file with an object like this:

{
  fieldname: 'photo',
  originalname: 'vacation.jpg',
  encoding: '7bit',
  mimetype: 'image/jpeg',
  destination: 'uploads/',
  filename: 'a1b2c3d4e5f6',       // random name, no extension
  path: 'uploads/a1b2c3d4e5f6',
  size: 245920                     // bytes
}

Notice the filename. Multer strips the original name and extension, replacing it with a random hex string. This is a security measure. If Multer kept the original filename, someone could upload a file called ../../etc/passwd or malicious.html and potentially cause problems depending on how you serve it. The random name prevents path traversal and naming conflicts.

The uploads/ directory needs to exist before you start the server. Multer does not create it for you. If the folder is missing, the upload fails with an error. Add this near the top of your server file:

const fs = require('fs');
const path = require('path');

const uploadDir = path.join(__dirname, 'uploads');
if (!fs.existsSync(uploadDir)) {
  fs.mkdirSync(uploadDir);
}

Handling multiple file uploads

Two scenarios here. First: multiple files from the same input field. Second: multiple file inputs with different names.

For the same field, use upload.array():

// accepts up to 5 files from a single input named "photos"
app.post('/gallery', upload.array('photos', 5), (req, res) => {
  console.log(req.files); // array of file objects
  console.log(req.files.length); // how many were uploaded
  res.json({ count: req.files.length });
});

<input type="file" name="photos" multiple />

The second argument to upload.array() is the maximum number of files. If someone tries to send 10 files when you set the limit to 5, Multer rejects the entire request.

For different fields, use upload.fields():

app.post('/profile', upload.fields([
  { name: 'avatar', maxCount: 1 },
  { name: 'resume', maxCount: 1 },
]), (req, res) => {
  console.log(req.files['avatar'][0]); // single file object
  console.log(req.files['resume'][0]); // single file object
  res.json({ message: 'got both files' });
});

With upload.fields(), req.files is an object where each key is the field name and each value is an array of file objects for that field. Even if maxCount is 1, you still get an array, so you access the file at index [0].

Controlling where and how files get saved

The dest: 'uploads/' shorthand works, but you get random filenames with no extensions. For anything beyond a quick prototype, you want more control. Multer's diskStorage engine lets you set the destination folder and the filename separately.

const storage = multer.diskStorage({
  destination: function (req, file, cb) {
    cb(null, 'uploads/');
  },
  filename: function (req, file, cb) {
    // keep original extension, prefix with timestamp to avoid collisions
    const uniqueName = Date.now() + '-' + file.originalname;
    cb(cb, uniqueName);
  },
});

const upload = multer({ storage: storage });

Wait, there is a bug in that code. I left it in on purpose because I made this exact mistake when I first wrote a storage config. The filename callback should be cb(null, uniqueName), not cb(cb, uniqueName). The first argument is an error (or null if no error). Passing the callback function itself as the error is a silent failure that produces a confusing error message about callbacks not being a function. Here is the corrected version:

const storage = multer.diskStorage({
  destination: function (req, file, cb) {
    cb(null, 'uploads/');
  },
  filename: function (req, file, cb) {
    const uniqueName = Date.now() + '-' + file.originalname;
    cb(null, uniqueName);
  },
});

const upload = multer({ storage: storage });

Now a file called vacation.jpg gets saved as something like 1717843200000-vacation.jpg. You keep the extension, avoid naming collisions, and can still identify the original file.

You should also filter what types of files you accept. Without a filter, someone can upload anything, executables included. Add a fileFilter function:

const upload = multer({
  storage: storage,
  limits: { fileSize: 5 * 1024 * 1024 }, // 5 MB max
  fileFilter: function (req, file, cb) {
    const allowed = ['image/jpeg', 'image/png', 'image/webp'];
    if (allowed.includes(file.mimetype)) {
      cb(null, true);
    } else {
      cb(new Error('Only JPEG, PNG, and WebP images are allowed'));
    }
  },
});

The limits option caps file size. The fileFilter checks the MIME type. If the file fails either check, Multer throws an error before writing anything to disk.

Handle Multer errors in your route so the client gets a readable response instead of a stack trace:

app.post('/upload', (req, res) => {
  upload.single('photo')(req, res, function (err) {
    if (err instanceof multer.MulterError) {
      // Multer-specific error (file too large, too many files, etc.)
      return res.status(400).json({ error: err.message });
    }
    if (err) {
      // your custom error from fileFilter
      return res.status(400).json({ error: err.message });
    }
    // no error, file is in req.file
    res.json({ file: req.file });
  });
});

Serving uploaded files back to users

Files sitting in an uploads/ folder are useless unless users can access them. The simplest way is to serve the folder as static files:

app.use('/uploads', express.static('uploads'));

Now a file saved as uploads/1717843200000-vacation.jpg is accessible at http://localhost:3000/uploads/1717843200000-vacation.jpg. You can use this URL in an <img> tag or send it back in an API response.

app.post('/upload', upload.single('photo'), (req, res) => {
  const fileUrl = `\({req.protocol}://\){req.get('host')}/uploads/${req.file.filename}`;
  res.json({
    message: 'uploaded',
    url: fileUrl,
  });
});

A word about production. Serving files directly from your Node server works for learning and small projects. Once you have real users and real traffic, you move uploaded files to a dedicated storage service like AWS S3 or Google Cloud Storage, and serve them through a CDN. The Node server should not be responsible for serving static files at scale. But that is a separate topic, and when you are starting out, express.static is the right tool.

A quick tip about .gitignore

Add your uploads folder to .gitignore. You do not want user-uploaded files committed to your repository. It bloats the repo, and if any of those files contain personal data, you have a privacy problem baked into your version history.

# .gitignore
uploads/

Keep the folder structure in your repo by adding an empty .gitkeep file inside uploads/ so Git tracks the directory without tracking its contents.

The full working example

Here is everything together in one file. In a real project you would split this into separate modules, but for learning it helps to see it all in one place.

const express = require('express');
const multer = require('multer');
const path = require('path');
const fs = require('fs');

const app = express();

// make sure uploads/ exists
const uploadDir = path.join(__dirname, 'uploads');
if (!fs.existsSync(uploadDir)) {
  fs.mkdirSync(uploadDir);
}

// storage config
const storage = multer.diskStorage({
  destination: function (req, file, cb) {
    cb(null, 'uploads/');
  },
  filename: function (req, file, cb) {
    cb(null, Date.now() + '-' + file.originalname);
  },
});

// upload instance with limits and filter
const upload = multer({
  storage: storage,
  limits: { fileSize: 5 * 1024 * 1024 },
  fileFilter: function (req, file, cb) {
    const allowed = ['image/jpeg', 'image/png', 'image/webp'];
    if (allowed.includes(file.mimetype)) {
      cb(null, true);
    } else {
      cb(new Error('Only JPEG, PNG, and WebP images are allowed'));
    }
  },
});

// serve uploaded files
app.use('/uploads', express.static('uploads'));

// single file upload
app.post('/upload', upload.single('photo'), (req, res) => {
  const fileUrl = `\({req.protocol}://\){req.get('host')}/uploads/${req.file.filename}`;
  res.json({ url: fileUrl });
});

// multiple files from same field
app.post('/gallery', upload.array('photos', 5), (req, res) => {
  const urls = req.files.map(f =>
    `\({req.protocol}://\){req.get('host')}/uploads/${f.filename}`
  );
  res.json({ urls });
});

app.listen(3000, () => console.log('running on port 3000'));

What to take away from this

Express does not handle file uploads on its own. The browser sends files using multipart/form-data, which is a different encoding than what Express parses by default. Multer fills that gap.

For a single file, use upload.single(). For multiple files from one input, use upload.array(). For files from different inputs, use upload.fields(). Always configure a fileFilter and a fileSize limit, even in development, because building that habit early saves you from accepting unexpected files later.

Start with express.static for serving uploads. Move to cloud storage when your project outgrows a single server. And keep your uploads out of Git.

References

• Multer documentation on npm

• Multer GitHub repository

• Express static files documentation

• MDN: multipart/form-data

• MDN: Using FormData objects

That is how I ended up learning about authentication. Not from a textbook or a course outline, but from the realization that my API had no front door.

What authentication actually means

Authentication is the server asking "who are you?" before doing anything. When you log into Instagram, you type your email and password. Instagram checks those against what it has stored. If they match, you are in. If they do not, you are not. That is authentication.

There is a related concept called authorization, which is "okay I know who you are, but are you allowed to do this?" A regular user can view posts. An admin can delete them. Authentication comes first. Authorization comes after. This post is about the first one.

The reason authentication matters for APIs specifically is that APIs do not have a login screen sitting in front of them. A browser shows you a form. An API just accepts HTTP requests from anywhere. Without authentication, every endpoint is open to anyone who knows the URL. Your delete route, your admin panel, your user data. All of it.

The old way: sessions

Before tokens became popular, most web apps used sessions. The flow worked like this: you log in, the server creates a session object and stores it somewhere (usually in memory or a database), and it sends you back a session ID as a cookie. Every time your browser makes a request, the cookie goes along automatically, and the server looks up the session to figure out who you are.

This works fine until you have multiple servers. If server A created the session and your next request hits server B, server B has no idea who you are because the session data lives on server A. You can fix this with a shared session store like Redis, but now you have another piece of infrastructure to manage.

Sessions are stateful. The server has to remember something about you between requests. That is the part that gets complicated at scale.

The stateless idea

JWTs take a different approach. Instead of the server remembering who you are, the server gives you a token after login that contains your identity information. On every request, you send the token back. The server reads the token, verifies it has not been tampered with, and knows who you are. No lookup needed. No session store. No shared state between servers.

The server does not remember anything. It just reads what is in the token and checks the signature. Stateless.

This is the core idea behind token-based authentication, and it is why JWTs became the default for APIs, single-page applications, and mobile apps. The server does less work per request, and scaling horizontally is simpler because any server can verify any token independently.

What a JWT actually looks like

JWT stands for JSON Web Token. When you see one, it looks like three chunks of gibberish separated by dots:

eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJ1c2VySWQiOiI2NWFiYzEyMyIsInJvbGUiOiJjdXN0b21lciIsImlhdCI6MTcwMDAwMDAwMCwiZXhwIjoxNzAwMDg2NDAwfQ.SflKxwRJSMeKKF2QT4fwpMeJf36POk6yJV_adQssw5c

Three parts. Each separated by a period. Each part is base64-encoded, which means it is not encrypted. Anyone can decode it. I will say that again because it surprised me: JWTs are not encrypted by default. They are encoded. There is a big difference, and I will get to why that matters.

The three parts

Header

The first chunk is the header. Decode it and you get:

{
  "alg": "HS256",
  "typ": "JWT"
}

Two fields. alg is the algorithm used to create the signature (more on that in a second). typ says it is a JWT. That is it. The header is boring. You will almost never think about it.

Payload

The second chunk is the payload. This is where the actual information lives:

{
  "userId": "65abc123",
  "role": "customer",
  "iat": 1700000000,
  "exp": 1700086400
}

userId and role are claims you put there yourself when creating the token. iat is "issued at," a Unix timestamp of when the token was created. exp is the expiration time. After that timestamp, the token is invalid.

You can put whatever you want in the payload. The user's name, their email, their permissions. But remember what I said about encoding versus encryption. Anyone who gets this token can decode the payload and read it. Do not put passwords, credit card numbers, or anything sensitive in here. Think of it like writing something on a postcard. Anyone handling it can read it. The signature just proves nobody changed the text.

Signature

The third chunk is the signature, and this is the part that actually matters for security. The server takes the header, the payload, and a secret key that only the server knows, and runs them through the algorithm specified in the header:

HMACSHA256(
  base64UrlEncode(header) + "." + base64UrlEncode(payload),
  your-secret-key
)

The output is the signature. When the server receives a token later, it repeats this exact calculation using the header and payload from the token plus its own secret key. If the result matches the signature in the token, the data has not been tampered with. If someone changed the role from customer to admin in the payload, the signature would not match, and the server would reject the token.

The secret key is the entire security model. If someone gets your secret key, they can forge valid tokens for any user. Keep it in an environment variable, never commit it to Git, and make it long and random.

The login flow

Here is what happens when a user logs in and then accesses a protected route. I am going to walk through every step because seeing the full picture once makes the individual pieces make more sense.

Step by step:

1. The client sends the email and password to a login endpoint.

2. The server looks up the user in the database and compares the password using bcrypt (never compare plain text, always hash).

3. If the password matches, the server creates a JWT containing the user's ID, role, and an expiration time.

4. The server sends the token back to the client.

5. The client stores the token (usually in localStorage or memory, depending on the app).

6. On every subsequent request, the client includes the token in the Authorization header.

7. The server reads the token, verifies the signature, checks expiration, and either processes the request or rejects it.

No session stored anywhere. No database lookup on every request to check who is logged in. The token itself carries the identity.

Building it

Enough theory. Let me show you the actual code, starting from an empty Express app with a MongoDB connection (covered in Part 5 of this series).

Install what you need

npm install express mongoose dotenv bcryptjs jsonwebtoken

bcryptjs is for hashing passwords. jsonwebtoken is the library that creates and verifies JWTs.

The User model

This is a trimmed version of the model from the Mongoose post. Password hashing happens in a pre-save hook so you never accidentally store a plain text password.

// models/User.js
const { Schema, model } = require('mongoose');
const bcrypt = require('bcryptjs');

const userSchema = new Schema({
  name: {
    type: String,
    required: [true, 'Name is required'],
    trim: true,
  },
  email: {
    type: String,
    required: [true, 'Email is required'],
    unique: true,
    lowercase: true,
    trim: true,
  },
  password: {
    type: String,
    required: [true, 'Password is required'],
    minlength: [8, 'Password must be at least 8 characters'],
    select: false, // never return password in queries by default
  },
  role: {
    type: String,
    enum: ['customer', 'admin'],
    default: 'customer',
  },
}, { timestamps: true });

// hash password before saving
userSchema.pre('save', async function(next) {
  if (!this.isModified('password')) return next();
  this.password = await bcrypt.hash(this.password, 12);
  next();
});

// compare password for login
userSchema.methods.comparePassword = async function(candidate) {
  return bcrypt.compare(candidate, this.password);
};

module.exports = model('User', userSchema);

The select: false on the password field means Mongoose will not include the password in query results unless you explicitly ask for it with .select('+password'). This prevents you from accidentally sending password hashes to the client in a response.

Generating a token

Create a small utility function that wraps the jsonwebtoken library. This keeps your route handlers clean and gives you one place to change token settings later.

// utils/generateToken.js
const jwt = require('jsonwebtoken');

function generateToken(user) {
  return jwt.sign(
    {
      userId: user._id,
      role: user.role,
    },
    process.env.JWT_SECRET,
    {
      expiresIn: '24h',
    }
  );
}

module.exports = generateToken;

jwt.sign() takes three arguments: the payload (what data to put in the token), the secret key, and an options object. expiresIn accepts human-readable strings like '24h', '7d', or '30m'. The library converts this into an exp claim in the payload automatically.

Your .env file needs a JWT_SECRET:

MONGO_URI=mongodb://localhost:27017/myapp
JWT_SECRET=a-long-random-string-that-nobody-can-guess-not-this-one-though

Generate a proper random secret in your terminal:

node -e "console.log(require('crypto').randomBytes(64).toString('hex'))"

Use the output of that command. Not "mysecret", not "jwt123", not "password". A short or guessable secret defeats the entire purpose of signing tokens.

The signup route

// routes/auth.js
const express = require('express');
const router = express.Router();
const User = require('../models/User');
const generateToken = require('../utils/generateToken');

router.post('/signup', async (req, res) => {
  try {
    const { name, email, password } = req.body;

    // check if user already exists
    const existing = await User.findOne({ email });
    if (existing) {
      return res.status(409).json({ error: 'Email already registered' });
    }

    const user = await User.create({ name, email, password });
    const token = generateToken(user);

    res.status(201).json({
      token,
      user: {
        id: user._id,
        name: user.name,
        email: user.email,
        role: user.role,
      },
    });
  } catch (err) {
    res.status(400).json({ error: err.message });
  }
});

Notice the response does not include user.password. Because of select: false in the schema, the password hash is not on the user object returned by User.create(). But even if it were, the response explicitly picks which fields to send. Be deliberate about what goes over the wire.

The login route

router.post('/login', async (req, res) => {
  try {
    const { email, password } = req.body;

    // .select('+password') overrides select: false for this query
    const user = await User.findOne({ email }).select('+password');
    if (!user) {
      return res.status(401).json({ error: 'Invalid email or password' });
    }

    const isMatch = await user.comparePassword(password);
    if (!isMatch) {
      return res.status(401).json({ error: 'Invalid email or password' });
    }

    const token = generateToken(user);

    res.json({
      token,
      user: {
        id: user._id,
        name: user.name,
        email: user.email,
        role: user.role,
      },
    });
  } catch (err) {
    res.status(500).json({ error: 'Login failed' });
  }
});

module.exports = router;

One thing to notice: the error message is the same for "email not found" and "wrong password." This is intentional. If you say "email not found," an attacker knows that email is not in your system. If you say "wrong password," they know the email is valid and can focus on brute-forcing the password. Same generic message for both prevents that information leak.

Sending the token with requests

After login, the client gets a token back. On every subsequent request to a protected endpoint, the client needs to send that token. The standard way is through the Authorization header using the Bearer scheme:

Authorization: Bearer eyJhbGciOiJIUzI1NiIs...

If you are testing with curl:

curl http://localhost:3000/api/orders \
  -H "Authorization: Bearer YOUR_TOKEN_HERE"

If you are building a frontend with fetch:

const token = localStorage.getItem('token');

const response = await fetch('/api/orders', {
  headers: {
    'Authorization': `Bearer ${token}`,
    'Content-Type': 'application/json',
  },
});

const data = await response.json();

If you are using axios, you can set a default header so you do not have to add it to every request:

const axios = require('axios');

// set once after login
axios.defaults.headers.common['Authorization'] = `Bearer ${token}`;

// now every request includes the token automatically
const res = await axios.get('/api/orders');

The word Bearer before the token is a convention defined in RFC 6750. It tells the server "I am authenticating by bearing (carrying) this token." You will see this in almost every API that uses JWTs.

Protecting routes with middleware

This is the part where the authentication actually happens on the server side. You write a middleware function that runs before your route handler. It checks for the token, verifies it, and either lets the request through or rejects it.

// middleware/auth.js
const jwt = require('jsonwebtoken');
const User = require('../models/User');

async function authenticate(req, res, next) {
  // 1. get the token from the header
  const authHeader = req.headers.authorization;
  if (!authHeader || !authHeader.startsWith('Bearer ')) {
    return res.status(401).json({ error: 'No token provided' });
  }

  const token = authHeader.split(' ')[1];

  try {
    // 2. verify the token
    const decoded = jwt.verify(token, process.env.JWT_SECRET);

    // 3. check if the user still exists
    const user = await User.findById(decoded.userId);
    if (!user) {
      return res.status(401).json({ error: 'User no longer exists' });
    }

    // 4. attach user to the request object
    req.user = user;
    next();
  } catch (err) {
    if (err.name === 'TokenExpiredError') {
      return res.status(401).json({ error: 'Token expired' });
    }
    return res.status(401).json({ error: 'Invalid token' });
  }
}

module.exports = authenticate;

Walking through this line by line:

First, it checks if the Authorization header exists and starts with Bearer . If not, the request is rejected immediately. No token, no access.

Then it pulls the token out by splitting on the space and taking the second part (index 1). "Bearer eyJhb..." becomes "eyJhb...".

jwt.verify() does two things at once. It checks the signature to make sure the token was not tampered with, and it checks if the token has expired. If either check fails, it throws an error.

The User.findById(decoded.userId) step is optional but I recommend it. The token might be valid but the user could have been deleted from the database since the token was issued. Without this check, a deleted user's token would still work until it expires.

Finally, it attaches the user to req.user. This means any route handler that runs after the middleware can access req.user without doing another database lookup.

Using the middleware

Apply it to any route that needs protection:

// routes/orders.js
const express = require('express');
const router = express.Router();
const authenticate = require('../middleware/auth');

// public route, no auth needed
router.get('/menu', async (req, res) => {
  // anyone can see the menu
  const items = await MenuItem.find();
  res.json(items);
});

// protected route
router.get('/orders', authenticate, async (req, res) => {
  // req.user is available because authenticate middleware set it
  const orders = await Order.find({ user: req.user._id });
  res.json(orders);
});

// protected route
router.post('/orders', authenticate, async (req, res) => {
  const order = await Order.create({
    user: req.user._id,
    items: req.body.items,
    totalAmount: req.body.totalAmount,
  });
  res.status(201).json(order);
});

The pattern is simple: add authenticate as a second argument before the route handler. It runs first. If the token is valid, next() passes control to the route handler and req.user is populated. If not, the middleware sends a 401 response and the route handler never executes.

Role-based access control

Once you have authentication working, adding authorization is a small step. You already have req.user.role available. Write another middleware that checks it:

// middleware/authorize.js
function authorize(...allowedRoles) {
  return (req, res, next) => {
    if (!req.user) {
      return res.status(401).json({ error: 'Not authenticated' });
    }

    if (!allowedRoles.includes(req.user.role)) {
      return res.status(403).json({ error: 'Not authorized' });
    }

    next();
  };
}

module.exports = authorize;

Use it after authenticate:

const authenticate = require('../middleware/auth');
const authorize = require('../middleware/authorize');

// only admins can delete menu items
router.delete(
  '/menu/:id',
  authenticate,
  authorize('admin'),
  async (req, res) => {
    await MenuItem.findByIdAndDelete(req.params.id);
    res.json({ message: 'Deleted' });
  }
);

// both admins and riders can update order status
router.patch(
  '/orders/:id/status',
  authenticate,
  authorize('admin', 'rider'),
  async (req, res) => {
    const order = await Order.findByIdAndUpdate(
      req.params.id,
      { status: req.body.status },
      { new: true }
    );
    res.json(order);
  }
);

The 401 versus 403 distinction matters. 401 means "I do not know who you are" (not authenticated). 403 means "I know who you are, but you are not allowed to do this" (not authorized). Mixing them up makes debugging harder for the person consuming your API.

Token expiration and why it matters

I set expiresIn: '24h' in the token generation function. That means after 24 hours, the token stops working and the user has to log in again. This seems annoying, and it is, but it exists for a good reason.

JWTs cannot be invalidated individually. Once a token is issued, it is valid until it expires. There is no "log this token out" button on the server because the server does not track tokens. If someone steals a token, the attacker has access until that token expires. A short expiration limits the damage window.

The tradeoff is user experience. A 15-minute token means users log in constantly. A 7-day token means a stolen token is useful for a week. Most APIs pick something in the middle, like 1 to 24 hours, and use refresh tokens for longer sessions. Refresh tokens are a deeper topic that I will cover in a separate post, but the basic idea is: the short-lived access token handles regular requests, and a longer-lived refresh token lets the client get a new access token without the user re-entering their password.

For now, 24 hours is a reasonable default while you are learning. Adjust based on what your app needs.

Common mistakes I made (and how to avoid them)

Storing tokens in localStorage without thinking about it

localStorage is the easy option, and for many apps it is fine. But localStorage is accessible to any JavaScript running on your page. If your site has an XSS vulnerability (a script injection), an attacker can read the token from localStorage and send it to their own server.

The more secure option for browser apps is an httpOnly cookie, which JavaScript cannot access at all. The browser sends it automatically with every request. The tradeoff is that cookies require CORS configuration and are vulnerable to CSRF attacks instead, which you mitigate with CSRF tokens. There is no perfect answer. For a learning project, localStorage is fine. For a production app with real users, research httpOnly cookies.

Using a weak secret

I used "secret" as my JWT_SECRET during development. If an attacker knows or guesses your secret, they can create valid tokens for any user. Use the crypto.randomBytes command I showed earlier. Make it at least 64 characters.

Putting sensitive data in the payload

I put a user's email in the payload once, which was fine. Then I put their phone number in there, which was less fine. Then I considered putting their address in there and realized I was heading in a bad direction. The payload is readable by anyone. Only put identifiers and roles in there, not personal information.

Not handling expired tokens on the frontend

When a token expires, the server returns 401. If your frontend does not handle this, the user sees a broken page with no explanation. Add a response interceptor (if using axios) or a wrapper around fetch that checks for 401 responses and redirects to the login page:

axios.interceptors.response.use(
  (response) => response,
  (error) => {
    if (error.response && error.response.status === 401) {
      localStorage.removeItem('token');
      window.location.href = '/login';
    }
    return Promise.reject(error);
  }
);

Testing the whole flow

Here is how to test everything end to end using curl. You can also use Postman or Thunder Client in VS Code, but curl works anywhere.

# 1. Sign up
curl -X POST http://localhost:3000/api/auth/signup \
  -H "Content-Type: application/json" \
  -d '{"name": "Priya", "email": "priya@example.com", "password": "mypassword123"}'

# response:
# {
#   "token": "eyJhbGciOiJIUzI1NiIs...",
#   "user": { "id": "...", "name": "Priya", "email": "priya@example.com", "role": "customer" }
# }

# 2. Copy the token from the response and use it:
TOKEN="eyJhbGciOiJIUzI1NiIs..."

# 3. Access a protected route
curl http://localhost:3000/api/orders \
  -H "Authorization: Bearer $TOKEN"

# 4. Try without a token
curl http://localhost:3000/api/orders
# { "error": "No token provided" }

# 5. Try with a garbage token
curl http://localhost:3000/api/orders \
  -H "Authorization: Bearer this.is.fake"
# { "error": "Invalid token" }

The full project structure

project/
  .env
  server.js
  db.js
  models/
    User.js
  routes/
    auth.js
    orders.js
  middleware/
    auth.js
    authorize.js
  utils/
    generateToken.js

// server.js
require('dotenv').config();
const express = require('express');
const connectDB = require('./db');
const authRoutes = require('./routes/auth');
const orderRoutes = require('./routes/orders');

const app = express();
app.use(express.json());

connectDB();

app.use('/api/auth', authRoutes);
app.use('/api', orderRoutes);

const PORT = process.env.PORT || 3000;
app.listen(PORT, () => {
  console.log(`server running on port ${PORT}`);
});

// db.js
const mongoose = require('mongoose');

async function connectDB() {
  try {
    await mongoose.connect(process.env.MONGO_URI);
    console.log('connected to MongoDB');
  } catch (err) {
    console.error('connection failed:', err.message);
    process.exit(1);
  }
}

module.exports = connectDB;

What I wish someone had told me

JWTs are not a complete solution. They handle the "prove who you are on each request" part well, but they do not handle logout (you cannot invalidate a token without maintaining state), they do not handle "this user changed their password so all their existing tokens should stop working" (same problem), and they do not handle refresh flows without extra work. For a learning project or an API where short token lifetimes are acceptable, JWTs are great. For a complex app with real security requirements, you will eventually layer more on top.

The thing that actually made authentication click for me was building it once from scratch, messing it up, locking myself out, and then understanding each piece by fixing it. The login route is just comparing a password and creating a signed string. The middleware is just reading that string back and checking the signature. The protected route is just checking if the middleware said "this person is real" before running. None of it is magic. It is just strings and if statements and a bit of cryptography you do not have to understand deeply to use correctly.

Start with the code in this post, get it running, then try breaking it. Change the secret key after creating a token and see what happens. Remove the exp field and think about what that means. Send a request without the Bearer prefix. The errors you get will teach you more than reading about it ever could.

References

• jsonwebtoken on npm - the library used throughout this post

• jwt.io - paste a token here to decode and inspect it visually, also has a debugger

• bcryptjs on npm - password hashing library

• RFC 7519: JSON Web Token - the official JWT specification, dense but complete

• RFC 6750: Bearer Token Usage - why the Authorization header uses the word "Bearer"

• OWASP JWT Cheat Sheet - security best practices for JWTs in production

• Mongoose documentation - for the User model and pre-save hooks

• Express middleware guide - how middleware works in Express

Turns out JavaScript escaped the browser in 2009. A developer named Ryan Dahl took Chrome's V8 engine, the thing that actually reads and runs your JavaScript code, and wrapped it in a program that could run on your operating system directly. No browser needed. That program is Node.js. You write JavaScript in a file, you run that file with the node command, and your operating system executes it the same way it would run a Python script or a C program. The language is the same. The environment is completely different.

The browser gives you document, window, alert(). Node gives you the filesystem, network access, the ability to start an HTTP server. Same syntax, different world underneath.

I want to walk through everything it takes to go from "I have never touched Node" to "I just wrote a working web server." No frameworks, no npm packages, just Node itself and what it ships with.

Installing Node.js

Go to nodejs.org. You will see two versions. One says LTS (Long Term Support), the other says Current. Pick LTS. Current has newer features but LTS is what companies actually use in production, and it gets security patches for longer. You are not going to need bleeding-edge features right now.

The installer works on Windows, macOS, and Linux. Download it, run it, click through the defaults. On Linux you can also install through your package manager, but the version in apt or yum is often outdated. The official installer or a version manager is a better bet.

If you are on macOS or Linux and want to manage multiple Node versions later, look into nvm. It lets you install and switch between Node versions with a single command. On Windows, nvm-windows does the same thing. You do not need this today, but knowing it exists saves future headaches.

# Install via nvm (macOS/Linux) if you want this route
curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.40.3/install.sh | bash
nvm install --lts
nvm use --lts

Or just use the installer from the website. Both work.

Checking that it actually installed

Open a terminal. On Windows that is PowerShell or Command Prompt. On macOS it is Terminal. On Linux, whatever terminal emulator you have.

node -v

v22.15.0

If you see a version number, Node is installed. If you see "command not found," the installer either failed or your terminal does not know where Node lives (a PATH issue, which I covered in the Linux post if you want the full story on what PATH actually is).

Check npm too:

npm -v

10.9.2

npm is Node's package manager. It comes bundled with Node automatically. You will use it later to install libraries, but for this post we are not touching it. Everything here uses only what Node ships with out of the box.

Your version numbers will probably be different from mine. That is fine. As long as both commands return a number and not an error, you are good.

The REPL: JavaScript without a file

Before writing any files, there is something worth trying. Type node in your terminal with no arguments:

node

Welcome to Node.js v22.15.0.
Type ".help" for more information.
>

That > prompt is the REPL. REPL stands for Read, Eval, Print, Loop. It reads what you type, evaluates it as JavaScript, prints the result, and loops back waiting for more input. It is an interactive JavaScript sandbox right in your terminal.

> 2 + 2
4
> "hello".toUpperCase()
'HELLO'
> const x = 10
undefined
> x * 3
30
> [1, 2, 3].map(n => n * 2)
[ 2, 4, 6 ]

Every line executes immediately. No file, no save, no refresh. The undefined you see after const x = 10 is just the REPL telling you that the assignment expression itself did not return a value. It is not an error.

The REPL is useful for quick experiments. You want to check how a string method works, or test whether your regex matches something, or try out an array operation before putting it in your code. Faster than opening a browser console, and it runs the exact same V8 engine.

A few REPL commands worth knowing:

.help     shows available commands
.exit     quits the REPL (Ctrl+C twice also works)
.editor   opens multi-line mode so you can paste or type a function

Try .editor mode:

> .editor
// Entering editor mode (Ctrl+D to finish, Ctrl+C to cancel)
function greet(name) {
  return `hey ${name}, welcome to Node`;
}
// press Ctrl+D
> greet("Saumya")
'hey Saumya, welcome to Node'

The REPL is a scratchpad. Use it when you want to test one thing quickly. For anything longer than a few lines, write a file.

To exit: type .exit or hit Ctrl+C twice.

Your first JavaScript file

Create a folder somewhere. Call it whatever you want. I will use node-basics.

mkdir node-basics
cd node-basics

Create a file called hello.js. You can use any text editor. VS Code, Sublime, Vim, Nano, Notepad, it does not matter. The file just needs to end in .js.

// hello.js
const name = "Saumya";
const hour = new Date().getHours();

let greeting;

if (hour < 12) {
  greeting = "Good morning";
} else if (hour < 17) {
  greeting = "Good afternoon";
} else {
  greeting = "Good evening";
}

console.log(`\({greeting}, \){name}.`);
console.log(`It is currently ${hour}:00 hours.`);
console.log("This is running outside the browser.");

Nothing fancy. Variables, a conditional, template literals, console.log. The same JavaScript you already know. The only difference is where it runs.

Running it

node hello.js

Good afternoon, Saumya.
It is currently 14:00 hours.
This is running outside the browser.

That is it. Node read your file, executed the JavaScript, printed the output to your terminal. No browser opened. No HTML page. Your operating system ran JavaScript the same way it runs any other program.

What just happened under the hood is worth understanding, even briefly.

When you type node hello.js, the Node.js runtime starts up. It hands your code to V8, which compiles the JavaScript into machine code that your CPU can actually execute. V8 does not interpret JavaScript line by line like some languages do. It compiles it. That is why Node is fast enough to run web servers handling thousands of requests.

The output goes to stdout, which is what your terminal is reading. console.log in Node writes to stdout. In the browser, console.log writes to the browser's developer console. Same function name, different destination.

Try adding an error to the file on purpose:

// error-test.js
console.log("this will print");
undefinedVariable.doSomething();
console.log("this will NOT print");

node error-test.js

this will print
/home/you/node-basics/error-test.js:2
undefinedVariable.doSomething();
^

ReferenceError: undefinedVariable is not defined
    at Object.<anonymous> (/home/you/node-basics/error-test.js:2:1)

Node gives you the file name, the line number, a caret pointing at the exact spot, and the error type. Read these. They tell you what went wrong and where. The first line printed because Node executes sequentially until it hits an unhandled error, then it stops. The third line never ran.

A few things Node has that the browser does not

Before we build the server, I want to show a couple of things that make Node different from browser JavaScript. You cannot access these in Chrome's console.

// node-extras.js

// __dirname: the absolute path to the folder this file is in
console.log("This file lives in:", __dirname);

// __filename: the absolute path to this file itself
console.log("This file is:", __filename);

// process: information about the running Node process
console.log("Node version:", process.version);
console.log("Operating system:", process.platform);
console.log("Current directory:", process.cwd());

// process.argv: command line arguments passed to the script
console.log("Arguments:", process.argv);

node node-extras.js hello world

This file lives in: /home/you/node-basics
This file is: /home/you/node-basics/node-extras.js
Node version: v22.15.0
Operating system: linux
Current directory: /home/you/node-basics
Arguments: [ '/usr/bin/node', '/home/you/node-basics/node-extras.js', 'hello', 'world' ]

process.argv is an array. The first element is always the path to the Node binary. The second is the path to your script. Everything after that is whatever you typed after the filename. This is how CLI tools read your input.

A quick practical use:

// greet-cli.js
const name = process.argv[2] || "stranger";
console.log(`Hello, ${name}.`);

node greet-cli.js Ravi

Hello, Ravi.

node greet-cli.js

Hello, stranger.

You just built a tiny CLI tool. Nothing installed, no libraries, just process.argv and a fallback.

Writing a Hello World server

This is the part where Node stops feeling like "JavaScript that runs in the terminal" and starts feeling like something genuinely different. We are going to write an HTTP server from scratch. No Express, no framework, just the http module that ships with Node.

// server.js
const http = require("http");

const server = http.createServer((req, res) => {
  res.writeHead(200, { "Content-Type": "text/plain" });
  res.end("Hello from Node.js\n");
});

server.listen(3000, () => {
  console.log("Server running at http://localhost:3000");
});

node server.js

Server running at http://localhost:3000

Open your browser and go to http://localhost:3000. You will see "Hello from Node.js" on the page. Your terminal is still running. The server is sitting there, waiting for requests.

Let me break down what each part does.

require("http") loads Node's built-in HTTP module. This module knows how to speak HTTP, the protocol your browser uses to talk to websites. You did not install this. It comes with Node.

http.createServer() creates a server object. The function you pass it runs every single time someone makes a request to your server. That function gets two arguments: req (the incoming request) and res (the response you send back).

res.writeHead(200, { "Content-Type": "text/plain" }) sets the HTTP status code to 200 (which means "OK, everything is fine") and tells the browser the response body is plain text.

res.end("Hello from Node.js\n") sends the actual response body and closes the connection. If you forget res.end(), the browser will hang forever waiting for the response to finish.

server.listen(3000) tells the server to start listening on port 3000. A port is just a number that identifies which program should receive incoming network traffic. Port 80 is the default for HTTP, port 443 for HTTPS. We use 3000 because it is unlikely to conflict with anything else running on your machine.

The callback in listen() runs once the server is ready. That is where the console.log fires.

Here is what happens when your browser visits localhost:3000:

To stop the server, go back to your terminal and press Ctrl+C. The process exits and the port is freed.

Making the server do more than one thing

A server that returns the same text for every URL is not very useful. Let's handle different routes:

// server-routes.js
const http = require("http");

const server = http.createServer((req, res) => {
  // req.url contains the path the browser requested
  // req.method contains GET, POST, etc.

  if (req.url === "/" && req.method === "GET") {
    res.writeHead(200, { "Content-Type": "text/html" });
    res.end("<h1>Home page</h1><p>Try visiting /about or /time</p>");

  } else if (req.url === "/about" && req.method === "GET") {
    res.writeHead(200, { "Content-Type": "text/html" });
    res.end("<h1>About</h1><p>This is a Node.js server with no framework.</p>");

  } else if (req.url === "/time" && req.method === "GET") {
    const now = new Date().toLocaleTimeString();
    res.writeHead(200, { "Content-Type": "application/json" });
    res.end(JSON.stringify({ time: now }));

  } else {
    res.writeHead(404, { "Content-Type": "text/plain" });
    res.end("404 - not found\n");
  }
});

server.listen(3000, () => {
  console.log("Server running at http://localhost:3000");
});

node server-routes.js

Now try these in your browser:

• http://localhost:3000/ returns an HTML page

• http://localhost:3000/about returns a different HTML page

• http://localhost:3000/time returns JSON with the current time

• http://localhost:3000/anything-else returns a 404

Notice the /time route sets Content-Type to application/json and uses JSON.stringify to turn a JavaScript object into a JSON string. This is how APIs work. The browser (or any HTTP client) gets back structured data it can parse and use.

The req.url and req.method check is the most primitive form of routing. In a real app you would use Express or Fastify to handle this, because writing if/else blocks for every URL gets old fast. But seeing it done manually first is how you understand what those frameworks actually do underneath. They are reading req.url and req.method and matching patterns, just with nicer syntax.

A habit worth building right now

Every time you modify your server code, you have to stop the server (Ctrl+C) and restart it (node server.js). This gets annoying within about ten minutes.

Node 22 has a built-in watch mode:

node --watch server.js

Now when you save changes to server.js, Node automatically restarts the server. No extra tools needed. If you are on an older Node version, nodemon is the classic npm package that does the same thing, but if your Node supports --watch, use that first.

What is actually different from browser JavaScript

By this point you have written and run JavaScript outside the browser. The syntax is identical, but the environment is not. Here is what changed:

The browser gives you window, document, localStorage, fetch (in modern browsers), alert(), and the entire DOM. None of these exist in Node. If you try document.getElementById in a Node script, you get a ReferenceError. There is no document. There is no HTML page. There is no DOM.

Node gives you process, require() (or import with ES modules), __dirname, __filename, access to the filesystem through the fs module, the ability to create servers with http, read and write files, run child processes, interact with your operating system. None of these exist in the browser. If you try require("fs") in Chrome's console, it will not work.

The language is one thing. The environment is another. People mix these up constantly when starting out. If a tutorial shows document.querySelector it is browser JavaScript. If it shows require("http") it is Node JavaScript. The JavaScript itself is identical. What changes is the stuff around it.

Common mistakes when you are starting out

I am putting these here because I made all of them.

Running node with no filename and wondering why your file did not execute. Typing node alone opens the REPL. Typing node server.js runs the file. They do different things.

Editing the file and expecting the running server to pick up changes. Node reads your file once when it starts. If you change the file, the running process does not know. Restart it, or use --watch.

Forgetting res.end() in your server. The browser will just keep spinning. The connection stays open, waiting for more data that never comes. Every response needs to be closed with res.end().

Using require for a file and forgetting the path prefix. require("http") loads a built-in module. require("myfile") tries to find a package called "myfile" in node_modules. require("./myfile") loads a file relative to the current file. The ./ matters.

Port already in use. If you see EADDRINUSE when starting your server, another process is already using that port. Either that process is still running from last time (check with lsof -i :3000 on macOS/Linux or netstat -ano | findstr :3000 on Windows), or pick a different port number.

Where to go from here

At this point you can install Node, run JavaScript files, use the REPL for quick tests, and write a basic HTTP server. That is enough to start building things.

The next post in this series covers npm and packages: what package.json is, how to install and use libraries, and the difference between dependencies and devDependencies. After that we will get into Express, which takes the manual routing we did here and makes it not terrible.

For now, the best thing you can do is experiment. Write scripts that read command-line arguments and do something with them. Add more routes to your server. Try returning different content types. Break things and read the error messages.

References

• Node.js official site - download and installation

• Node.js docs: HTTP module - the module we used to build the server

• Node.js docs: Process - everything about the process object

• nvm (Node Version Manager) - manage multiple Node versions on macOS/Linux

• nvm-windows - same thing for Windows

• V8 JavaScript engine - the engine inside both Chrome and Node

• MDN: JavaScript reference - when you need to look up any JS syntax

If you are coming from Python or Java or really anything where code runs line by line and waits for each thing to finish before moving on, Node.js will confuse you. It confused me. I wrote what looked like perfectly reasonable code, and the output came back empty. The file existed. The path was correct. The data just was not there yet when my code tried to use it.

This post is about why that happens, what callbacks are, why they get ugly fast, and how promises fix the mess.

Why Node.js does not wait

Most programming languages are synchronous by default. You say "read this file" and the program stops, waits for the disk to return the data, then moves to the next line. Simple. Predictable.

Node.js does not do that. When you tell Node to read a file, it sends that request to the operating system and immediately moves to the next line of code. It does not wait. The file might take 5 milliseconds or 500 milliseconds to read, and Node does not care. It has other things to do.

This is called non-blocking I/O, and the reason Node works this way is that it runs on a single thread. One thread. If it stopped and waited for every file read, every database query, every HTTP request, it could only do one thing at a time. A web server handling 1000 users would grind to a halt because user 2 is waiting for user 1's database query to finish.

So Node keeps moving. It fires off the I/O operation, continues executing whatever comes next, and when the result comes back, it runs a function you gave it earlier. That function is a callback.

Here is what it looks like when you do not account for this:

const fs = require('fs');

let content;
fs.readFile('greeting.txt', 'utf8', (err, data) => {
  content = data;
});

console.log(content);

undefined

The console.log runs before readFile finishes. By the time the callback fires and assigns data to content, the log is long gone. The variable was undefined at the moment you printed it. The data showed up a few milliseconds later, but nobody was listening anymore.

This is the first wall every beginner hits with Node. Your code runs out of order.

Callbacks: the original solution

A callback is a function you hand to another function and say "call this when you are done." Instead of waiting for a result, you describe what should happen with the result whenever it arrives.

const fs = require('fs');

fs.readFile('greeting.txt', 'utf8', (err, data) => {
  if (err) {
    console.error('Failed to read file:', err.message);
    return;
  }
  console.log(data);
});

console.log('This prints first');

This prints first
Hello from greeting.txt!

The third argument to readFile is the callback. Node reads the file in the background, and when it finishes, it calls your function with two arguments: an error (if something went wrong) and the data (if it worked). This pattern of (err, data) as the callback signature is called "error-first callbacks" and it is everywhere in older Node code.

The if (err) check at the top is not optional. If you skip it and the file does not exist, data is undefined and your code breaks in a confusing way downstream instead of failing clearly right where the problem is.

Let me walk through the flow of what actually happens at runtime:

1. Node sees fs.readFile() — sends read request to OS, moves on
2. Node sees console.log('This prints first') — prints it
3. Node has nothing left to do — enters the event loop, waits
4. OS finishes reading the file — puts callback in the queue
5. Event loop picks up the callback — runs your function
6. console.log(data) prints the file contents

The event loop is the mechanism that checks "are there any callbacks waiting to run?" in a continuous cycle. When your main code finishes executing, the event loop takes over and processes whatever I/O results have come back.

The staircase problem

One callback is fine. Two nested callbacks are tolerable. Three gets uncomfortable. The task I mentioned at the start, reading three files and combining them, looks like this with callbacks:

const fs = require('fs');

fs.readFile('header.txt', 'utf8', (err1, header) => {
  if (err1) {
    console.error('header failed:', err1.message);
    return;
  }
  fs.readFile('body.txt', 'utf8', (err2, body) => {
    if (err2) {
      console.error('body failed:', err2.message);
      return;
    }
    fs.readFile('footer.txt', 'utf8', (err3, footer) => {
      if (err3) {
        console.error('footer failed:', err3.message);
        return;
      }
      const combined = header + '\n' + body + '\n' + footer;
      fs.writeFile('page.txt', combined, (err4) => {
        if (err4) {
          console.error('write failed:', err4.message);
          return;
        }
        console.log('page.txt written');
      });
    });
  });
});

Look at the shape of that code. Every step indents further right. Four levels deep for something that is, conceptually, four sequential steps. This is what people call callback hell, and the name is earned. The logic is correct but the structure makes it genuinely hard to follow, hard to modify, and hard to handle errors consistently.

Try adding a fifth step in the middle of that. Try moving the order of operations around. Try figuring out which closing brace belongs to which callback. I spent a while counting braces and parentheses before accepting that there had to be a better way.

The error handling is especially painful. Each callback checks for its own error separately. There is no single place to catch failures. If you forget the error check in one callback, that error silently vanishes and the next step runs with undefined data.

Promises: the better way

A Promise is an object that represents a value you do not have yet but will have later. It has three states: pending (still waiting), fulfilled (got the value), or rejected (something went wrong).

Promise states:

  PENDING ——> FULFILLED (resolved with a value)
      |
      +——> REJECTED (failed with an error)

Once settled (fulfilled or rejected), a Promise never changes state again.

Node's fs module has a promise-based version built in. You get it from fs/promises:

const fs = require('fs/promises');

fs.readFile('greeting.txt', 'utf8')
  .then(data => {
    console.log(data);
  })
  .catch(err => {
    console.error('Failed:', err.message);
  });

No callback argument. readFile returns a Promise. You call .then() on it to say what happens when it succeeds, and .catch() to say what happens when it fails. The error handling is in one place instead of scattered inside every callback.

Now here is the same three-file task with promises:

const fs = require('fs/promises');

let headerText, bodyText;

fs.readFile('header.txt', 'utf8')
  .then(header => {
    headerText = header;
    return fs.readFile('body.txt', 'utf8');
  })
  .then(body => {
    bodyText = body;
    return fs.readFile('footer.txt', 'utf8');
  })
  .then(footer => {
    const combined = headerText + '\n' + bodyText + '\n' + footer;
    return fs.writeFile('page.txt', combined);
  })
  .then(() => {
    console.log('page.txt written');
  })
  .catch(err => {
    console.error('Something failed:', err.message);
  });

No staircase. The code reads top to bottom. Each .then() returns a new promise, so they chain instead of nest. And the .catch() at the bottom handles errors from any step in the chain. If body.txt fails to read, the chain skips straight to .catch() and you get one clear error message.

This is the real win. One .catch() at the end replaces four separate if (err) checks.

But we can do better. If the three files do not depend on each other, there is no reason to read them one after another. Promise.all runs them at the same time:

const fs = require('fs/promises');

Promise.all([
  fs.readFile('header.txt', 'utf8'),
  fs.readFile('body.txt', 'utf8'),
  fs.readFile('footer.txt', 'utf8'),
])
  .then(([header, body, footer]) => {
    return fs.writeFile('page.txt', header + '\n' + body + '\n' + footer);
  })
  .then(() => console.log('page.txt written'))
  .catch(err => console.error('Failed:', err.message));

Three file reads happening in parallel, results collected into an array in the same order you passed them. If any one fails, the whole thing goes to .catch(). This is both shorter and faster than the sequential version.

Making your own promises

Sometimes you are working with older libraries that only support callbacks. You can wrap them in a promise yourself:

function readFilePromise(path) {
  return new Promise((resolve, reject) => {
    const fs = require('fs');
    fs.readFile(path, 'utf8', (err, data) => {
      if (err) reject(err);
      else resolve(data);
    });
  });
}

readFilePromise('greeting.txt')
  .then(data => console.log(data))
  .catch(err => console.error(err.message));

The new Promise() constructor takes a function with two arguments: resolve (call this with the value when it works) and reject (call this with the error when it fails). Once you call either one, the promise settles and .then() or .catch() fires accordingly.

Node also has util.promisify which does this wrapping for you on any function that follows the error-first callback pattern:

const { promisify } = require('util');
const fs = require('fs');

const readFile = promisify(fs.readFile);

readFile('greeting.txt', 'utf8')
  .then(data => console.log(data))
  .catch(err => console.error(err.message));

One line to convert any callback-based function into a promise-based one. This is how a lot of codebases transitioned from callbacks to promises without rewriting everything.

A few things that tripped me up

Forgetting to return inside .then(). If you start an async operation inside a .then() block but do not return it, the chain does not wait for it. The next .then() fires immediately with undefined. This is a quiet bug that does not throw errors, it just produces wrong results.

// broken — missing return
.then(header => {
  fs.readFile('body.txt', 'utf8'); // floats away, nobody waits for it
})
.then(body => {
  console.log(body); // undefined
})

// fixed
.then(header => {
  return fs.readFile('body.txt', 'utf8');
})
.then(body => {
  console.log(body); // actual file contents
})

The other thing: .catch() only catches errors from promises above it in the chain. If you put .catch() in the middle and then add more .then() calls after it, the chain continues after the catch. Sometimes that is what you want. Usually it is not, and having .catch() at the very end is the safer default.

What to actually use in 2025

If you are writing new code, use async/await. It is built on top of promises and reads like synchronous code. But you need to understand promises first because async/await is just syntax over them. When you see a confusing await error or need to run things in parallel with Promise.all, the promise knowledge is what lets you debug it.

For reference, here is the same three-file task with async/await as a preview:

const fs = require('fs/promises');

async function buildPage() {
  try {
    const [header, body, footer] = await Promise.all([
      fs.readFile('header.txt', 'utf8'),
      fs.readFile('body.txt', 'utf8'),
      fs.readFile('footer.txt', 'utf8'),
    ]);
    await fs.writeFile('page.txt', header + '\n' + body + '\n' + footer);
    console.log('page.txt written');
  } catch (err) {
    console.error('Failed:', err.message);
  }
}

buildPage();

That reads like normal code. No .then() chains, no nesting, try/catch instead of .catch(). But under the hood, every await is a promise. The next post will break that apart properly.

References

• Node.js docs: fs/promises

• MDN: Promise

• MDN: Using promises

• Node.js docs: util.promisify

• MDN: Promise.all

• Node.js event loop explained

Then someone explained REST to me in about ten minutes, and suddenly half the confusing stuff I had been copying from tutorials made sense. So that is what this post is.

APIs are just agreements

When your frontend needs data, it sends a request to your backend. The backend sends a response. That exchange is an API call. API stands for Application Programming Interface, which sounds complicated but the idea is boring: two programs agreed on how to talk to each other.

Think of it like ordering food. You do not walk into the kitchen and start cooking. You talk to the waiter. You say what you want in a format they understand (the menu), they bring back what you ordered. The waiter is the API. The kitchen is the server. You are the client.

REST is one specific set of rules for how that conversation should work. It stands for Representational State Transfer. The name is not helpful. What matters is the pattern: you use standard HTTP methods (GET, POST, PUT, DELETE) to perform actions on resources, and the server responds with data and a status code telling you what happened.

Most APIs you will use as a beginner follow REST conventions. Once you learn the pattern, every new API you encounter feels familiar.

Resources: the nouns of your API

In REST, everything revolves around resources. A resource is any piece of data your application manages. Users, posts, comments, products, orders. Each one is a resource.

The way you identify a resource is through its URL (or more formally, its URI). And the naming convention is simple: use nouns, not verbs. Plural nouns.

/users          (the whole collection of users)
/users/42       (one specific user, ID 42)
/posts          (all posts)
/posts/7        (post number 7)

You do not put actions in the URL. No /getUser, no /deletePost, no /createNewComment. The HTTP method (GET, POST, PUT, DELETE) already tells the server what action to perform. The URL just says what thing you are acting on.

This is the part that clicked for me late. I kept naming routes like /api/getUserById and /api/createUser because that felt natural. It works, but it falls apart once your app has 30 endpoints and none of them follow a pattern. REST gives you the pattern.

HTTP methods: the verbs

There are four methods you will use constantly. Each one maps to a CRUD operation (Create, Read, Update, Delete), which is the set of basic actions almost every application needs.

That is the whole mapping. Four methods, four operations. Let me walk through each one with actual Express code, because seeing the request and response together is what made this real for me.

GET: give me the data

GET requests fetch data. They should never change anything on the server. If a GET request modifies data, something is wrong with your design. This is called being "safe" in REST terminology.

// GET /users - fetch all users
app.get('/users', async (req, res) => {
  const users = await User.find();
  res.status(200).json(users);
});

// GET /users/42 - fetch one user
app.get('/users/:id', async (req, res) => {
  const user = await User.findById(req.params.id);
  if (!user) {
    return res.status(404).json({ error: 'User not found' });
  }
  res.status(200).json(user);
});

The response for a list of users looks like:

[
  { "_id": "abc123", "name": "Priya", "email": "priya@example.com" },
  { "_id": "def456", "name": "Ravi", "email": "ravi@example.com" }
]

Notice the route /users/:id. The colon means id is a parameter. When someone hits /users/42, Express puts "42" into req.params.id. You use that to look up one specific resource.

POST: create something new

POST sends data to the server to create a new resource. The data goes in the request body, not the URL.

// POST /users - create a new user
app.post('/users', async (req, res) => {
  try {
    const user = await User.create(req.body);
    res.status(201).json(user);
  } catch (err) {
    res.status(400).json({ error: err.message });
  }
});

The request body (what the client sends) would look like:

{
  "name": "Meera",
  "email": "meera@example.com",
  "age": 24
}

And the response comes back with the created user, including the _id that MongoDB generated:

{
  "_id": "ghi789",
  "name": "Meera",
  "email": "meera@example.com",
  "age": 24,
  "createdAt": "2025-01-15T10:30:00Z"
}

Status 201 means "created." Not 200. This is a small detail that a lot of beginners skip, and it matters because status codes tell the client exactly what happened without parsing the response body.

PUT: replace something that exists

PUT updates an existing resource. You send the full updated version, and the server replaces what was there before.

// PUT /users/42 - update user 42
app.put('/users/:id', async (req, res) => {
  const user = await User.findByIdAndUpdate(
    req.params.id,
    req.body,
    { new: true, runValidators: true }
  );
  if (!user) {
    return res.status(404).json({ error: 'User not found' });
  }
  res.status(200).json(user);
});

{ new: true } tells Mongoose to return the document after the update, not before. Without it you get back the old version, which is confusing the first time it happens.

There is also PATCH, which updates only specific fields instead of replacing the whole document. In practice, many APIs use PUT and PATCH interchangeably, which is technically wrong but extremely common. When you are starting out, PUT is fine. You can worry about the distinction later.

DELETE: remove something

DELETE removes a resource. The response usually confirms what was deleted or just returns an empty success.

// DELETE /users/42 - delete user 42
app.delete('/users/:id', async (req, res) => {
  const user = await User.findByIdAndDelete(req.params.id);
  if (!user) {
    return res.status(404).json({ error: 'User not found' });
  }
  res.status(200).json({ message: 'User deleted' });
});

Some APIs return 204 (No Content) with an empty body on successful deletion. Either approach is fine as long as you pick one and stick with it across your whole API.

Status codes: what the numbers mean

You have already seen a few of these. Status codes are three-digit numbers the server sends back to tell the client how things went. You do not need to memorize all of them. The groupings are what matter.

The ones you will use in almost every project:

200 means OK. The request succeeded. Used for GET, PUT, and DELETE responses.

201 means Created. A new resource was made. Used after a successful POST.

400 means Bad Request. The client sent something invalid, like missing required fields or a malformed email.

401 means Unauthorized. The client is not logged in or the auth token is missing.

403 means Forbidden. The client is logged in but does not have permission for this action.

404 means Not Found. The resource does not exist at that URL.

500 means Internal Server Error. Something crashed on the server. The client did nothing wrong.

A quick way to think about 401 vs 403: 401 is "who are you?" and 403 is "I know who you are, and no."

Route design: putting it together

Here is the full set of routes for a users resource, laid out the way a REST API expects them:

GET    /api/users          Fetch all users
GET    /api/users/:id      Fetch one user by ID
POST   /api/users          Create a new user
PUT    /api/users/:id      Update a user by ID
DELETE /api/users/:id      Delete a user by ID

Five routes. One resource. Consistent naming. If you add a posts resource later, the pattern is identical:

GET    /api/posts
GET    /api/posts/:id
POST   /api/posts
PUT    /api/posts/:id
DELETE /api/posts/:id

You can nest resources too. If posts belong to users:

GET    /api/users/:userId/posts       All posts by user
POST   /api/users/:userId/posts       Create a post for a user

Keep nesting to one level deep. /api/users/:userId/posts/:postId/comments/:commentId is technically valid but miserable to work with. If you find yourself going three levels deep, flatten it out.

A complete working example

This is all five user routes in one file, using Express and Mongoose, with proper status codes and error handling. You can drop this into a project and it works.

const express = require('express');
const router = express.Router();
const User = require('../models/User');

// GET /api/users
router.get('/', async (req, res) => {
  try {
    const users = await User.find().select('name email role');
    res.status(200).json(users);
  } catch (err) {
    res.status(500).json({ error: 'Failed to fetch users' });
  }
});

// GET /api/users/:id
router.get('/:id', async (req, res) => {
  try {
    const user = await User.findById(req.params.id);
    if (!user) return res.status(404).json({ error: 'User not found' });
    res.status(200).json(user);
  } catch (err) {
    if (err.name === 'CastError') {
      return res.status(400).json({ error: 'Invalid user ID format' });
    }
    res.status(500).json({ error: 'Failed to fetch user' });
  }
});

// POST /api/users
router.post('/', async (req, res) => {
  try {
    const user = await User.create(req.body);
    res.status(201).json(user);
  } catch (err) {
    if (err.name === 'ValidationError') {
      const messages = Object.values(err.errors).map(e => e.message);
      return res.status(400).json({ error: messages.join(', ') });
    }
    if (err.code === 11000) {
      return res.status(409).json({ error: 'Email already exists' });
    }
    res.status(500).json({ error: 'Failed to create user' });
  }
});

// PUT /api/users/:id
router.put('/:id', async (req, res) => {
  try {
    const user = await User.findByIdAndUpdate(
      req.params.id,
      req.body,
      { new: true, runValidators: true }
    );
    if (!user) return res.status(404).json({ error: 'User not found' });
    res.status(200).json(user);
  } catch (err) {
    res.status(400).json({ error: err.message });
  }
});

// DELETE /api/users/:id
router.delete('/:id', async (req, res) => {
  try {
    const user = await User.findByIdAndDelete(req.params.id);
    if (!user) return res.status(404).json({ error: 'User not found' });
    res.status(200).json({ message: 'User deleted' });
  } catch (err) {
    res.status(500).json({ error: 'Failed to delete user' });
  }
});

module.exports = router;

In your main server.js, mount it like this:

const userRoutes = require('./routes/users');
app.use('/api/users', userRoutes);

Now every route inside the file is prefixed with /api/users automatically. The router.get('/') becomes GET /api/users, and router.get('/:id') becomes GET /api/users/:id. Clean.

Things I wish someone told me earlier

Keep your route files separate from your logic. The route handler should be short. If your POST /users handler is 60 lines long with validation, database calls, email sending, and logging all mixed together, pull the logic into a controller or service function. The route file should read like a table of contents.

Use express.json() middleware. Without it, req.body is undefined on POST and PUT requests because Express does not parse JSON by default.

app.use(express.json());

I forgot this once and spent 40 minutes wondering why my request body was always empty. The server was fine. The database was fine. The client was sending data. Express just was not reading it.

Test your API with a tool like Postman, Insomnia, or even curl from the terminal. Testing through the browser only works for GET requests. You need a tool that lets you set the HTTP method, add headers, and include a JSON body.

# quick test from the terminal
curl -X POST http://localhost:3000/api/users \
  -H "Content-Type: application/json" \
  -d '{"name": "Priya", "email": "priya@example.com"}'

References

• MDN Web Docs: HTTP request methods

• MDN Web Docs: HTTP response status codes

• Express.js routing guide

• REST API design best practices (Stack Overflow blog)

• Postman: API testing tool

I had written blocking code without knowing what that meant.

This took me an embarrassingly long time to understand, partly because every explanation I found started with the event loop and callback queues and microtask priorities. I did not need any of that yet. I needed to understand one thing: when your code blocks, your entire server stops.

What blocking code actually looks like

Blocking means the program stops on one line and refuses to move to the next line until that operation finishes. The CPU sits there, waiting. Nothing else runs.

Here is the Node.js version of shooting yourself in the foot:

const fs = require('fs');

console.log('before reading file');

const data = fs.readFileSync('./bigfile.json', 'utf-8');

console.log('after reading file');
console.log('doing other stuff');

Output:

before reading file
// ... long pause while the file loads ...
after reading file
doing other stuff

readFileSync. That Sync at the end is the giveaway. It means synchronous. The program reaches that line, stops everything, waits for the entire file to load into memory, and only then moves on. If that file is 500MB, every line of code below it waits for 500MB to finish loading. If another user hits your server during that wait, they get nothing. The server is busy staring at a hard drive.

For a script you run once on your own machine, this is fine. For a server handling multiple users, it is a problem.

The restaurant analogy

Think about a restaurant with one waiter. A customer orders a steak. The waiter walks to the kitchen, hands in the order, and then stands there watching the chef cook. Does not take any other orders, does not refill anyone's water, does not greet new customers walking in. Just stands at the kitchen window, arms crossed, waiting for the steak.

That is blocking.

Now picture the same waiter, same restaurant. Customer orders a steak. The waiter hands the order to the kitchen and immediately walks back to the floor. Takes another order. Refills a drink. Seats a new table. When the kitchen rings the bell to say the steak is ready, the waiter picks it up and delivers it.

That is non-blocking.

The kitchen still takes the same amount of time to cook. The steak is not faster. But the waiter is not frozen in place while waiting for it, so every other customer in the restaurant gets served in the meantime.

Node.js is that single waiter. It has one thread. If you make it stand around waiting for a file to load or a database to respond, nobody else gets served.

What non-blocking code looks like

The non-blocking version of the same file read uses a callback:

const fs = require('fs');

console.log('before reading file');

fs.readFile('./bigfile.json', 'utf-8', (err, data) => {
  if (err) {
    console.error('read failed:', err.message);
    return;
  }
  console.log('file loaded, length:', data.length);
});

console.log('not waiting around');
console.log('doing other stuff');

Output:

before reading file
not waiting around
doing other stuff
file loaded, length: 4893722

Look at the order. "not waiting around" prints before the file finishes loading. The program did not stop. It told the operating system "read this file and call me back when you are done," then immediately moved to the next line. When the file finished loading, the callback function ran.

The file still took the same amount of time to read. But the rest of the program did not freeze while it happened.

Why this matters for servers

Here is where it gets real. Put blocking code inside a route handler and watch what happens to every other request.

const express = require('express');
const fs = require('fs');
const app = express();

// bad: blocking route
app.get('/data', (req, res) => {
  const data = fs.readFileSync('./bigfile.json', 'utf-8');
  res.json(JSON.parse(data));
});

// this route works fine on its own
app.get('/health', (req, res) => {
  res.json({ status: 'ok' });
});

app.listen(3000);

If someone hits /data and the file takes 3 seconds to read, every request to /health during those 3 seconds also waits 3 seconds. The health check has nothing to do with the file. It should return instantly. But Node.js has one thread, and that thread is stuck on readFileSync. Everybody waits.

The fix:

// good: non-blocking route
app.get('/data', async (req, res) => {
  try {
    const data = await fs.promises.readFile('./bigfile.json', 'utf-8');
    res.json(JSON.parse(data));
  } catch (err) {
    res.status(500).json({ error: 'could not read file' });
  }
});

Same result. Same file. But now while the file loads, Node.js goes back to handling other requests. The /health route responds instantly regardless of what /data is doing.

The timeline, visually

Here is what happens with blocking code when two requests come in:

User2 asked for a health check. A response that takes 0 milliseconds to generate. They waited 3 seconds because the server was stuck reading a file for someone else.

Now the non-blocking version:

User2 gets their response immediately. User1 still waits 3 seconds for the file, because the file is still big. But they are no longer holding the entire server hostage.

Callbacks, promises, async/await

Node.js has gone through three eras of writing non-blocking code. All three still work. You will see all three in other people's codebases.

Callbacks came first. You pass a function that runs when the async work finishes:

fs.readFile('./config.json', 'utf-8', (err, data) => {
  if (err) {
    console.error(err);
    return;
  }
  console.log(data);
});

The problem with callbacks shows up when you need to do three async things in sequence. Read a file, then use its contents to query a database, then use that result to send an email. Each step nests inside the previous callback. The code drifts to the right and becomes genuinely hard to follow. People call this callback hell, and the name is earned.

// callback hell
fs.readFile('./config.json', 'utf-8', (err, config) => {
  if (err) return handleError(err);
  db.query(config.query, (err, rows) => {
    if (err) return handleError(err);
    sendEmail(rows[0].email, 'hello', (err, result) => {
      if (err) return handleError(err);
      console.log('done');
    });
  });
});

Promises fixed the nesting:

fs.promises.readFile('./config.json', 'utf-8')
  .then(config => db.query(JSON.parse(config).query))
  .then(rows => sendEmail(rows[0].email, 'hello'))
  .then(result => console.log('done'))
  .catch(err => handleError(err));

Flat. Each .then() returns a new promise. Errors fall through to .catch(). Easier to read, easier to reason about.

Async/await made it look like regular code:

async function handleRequest() {
  try {
    const raw = await fs.promises.readFile('./config.json', 'utf-8');
    const config = JSON.parse(raw);
    const rows = await db.query(config.query);
    await sendEmail(rows[0].email, 'hello');
    console.log('done');
  } catch (err) {
    handleError(err);
  }
}

This is still non-blocking. The await keyword does not freeze the server the way readFileSync does. When execution hits await, Node.js pauses that specific function and goes to handle other work. When the awaited operation finishes, it comes back and picks up where it left off. Other requests keep flowing.

If you are starting fresh, use async/await. It reads like sequential code but behaves like non-blocking code. The callbacks and promise chains are worth recognizing because you will run into them in older projects, but for new code, async/await is the standard.

A real-world example: database calls

File reading is one thing, but the same blocking problem applies to anything that takes time. Database queries are the most common culprit in real applications.

// imagine a route that fetches user data and their orders
app.get('/user/:id/summary', async (req, res) => {
  try {
    const user = await User.findById(req.params.id);
    const orders = await Order.find({ user: req.params.id }).limit(10);

    res.json({
      name: user.name,
      email: user.email,
      recentOrders: orders,
    });
  } catch (err) {
    res.status(500).json({ error: 'something broke' });
  }
});

Both findById and find are async. While MongoDB processes the query, Node.js handles other requests. No one is stuck waiting.

But notice something: the user query finishes before the orders query starts. They do not depend on each other. You could run them at the same time:

app.get('/user/:id/summary', async (req, res) => {
  try {
    const [user, orders] = await Promise.all([
      User.findById(req.params.id),
      Order.find({ user: req.params.id }).limit(10),
    ]);

    res.json({
      name: user.name,
      email: user.email,
      recentOrders: orders,
    });
  } catch (err) {
    res.status(500).json({ error: 'something broke' });
  }
});

Promise.all fires both queries at the same time and waits for both to finish. If the user query takes 50ms and the orders query takes 80ms, the total wait is 80ms instead of 130ms. For two queries the difference is small. For five or six independent async operations, it adds up fast.

Common mistakes and how to avoid them

I am putting these here because I made every single one of them.

Using Sync methods in a server. Any function ending in Sync blocks the entire process. readFileSync, writeFileSync, execSync. These are fine in a CLI script or a build tool. They have no place in a server handling requests. If you see Sync in server code, replace it with the async version.

Forgetting try/catch around await. An unhandled promise rejection used to just log a warning. In newer versions of Node.js it crashes the process. Wrap your awaits in try/catch or use an error-handling middleware in Express.

Running independent async operations in sequence when they could run in parallel. If operation B does not depend on the result of operation A, use Promise.all. The code runs faster with no extra complexity.

Not understanding that await pauses the function, not the server. I see beginners avoid await because they think it "blocks." It does not. It pauses that one function and frees the thread for everything else. That is the whole point. Use it.

What to read next

If you want to go deeper on how Node.js actually handles all of this under the hood, here are solid resources.

• Node.js docs: blocking vs non-blocking gives the official explanation with examples

• Node.js docs: the event loop covers what happens behind the scenes when you write async code

• MDN: async/await walks through promises and async/await with examples

• MDN: using promises if you want to understand the promise chain pattern

HTTP has no memory. Every request your browser sends is a stranger knocking on a door. The server does not know if this is the same person who logged in ten seconds ago or someone else entirely. The protocol was built in 1991 to serve documents, not to remember people. That design choice is the reason sessions, cookies, and tokens exist.

The real question is not "how do I check a password." It is: after I check the password once, how do I avoid asking again on every single request?

Browser                              Server
  |                                    |
  |--- POST /login (email, pass) ----->|
  |                                    | checks password... correct
  |<--- 200 OK, here is your data -----|
  |                                    |
  |--- GET /dashboard --------------->|
  |                                    | who is this? no idea.
  |<--- 401 Unauthorized --------------|

The login worked. The very next request fails because HTTP forgets.

Two families of solutions exist. One says the server remembers you. The other says the server gives you a signed note that proves who you are, and you carry it around. That is the session vs JWT split, and everything else is details.

Cookies: the envelope, not the letter

People mix up cookies with sessions constantly. A cookie is just a transport mechanism. It is a small piece of text that a server sends to a browser, and the browser automatically sends it back on every future request to that server. The cookie does not know what it is carrying. It could hold a session ID, a JWT, or your preferred theme color.

res.cookie('sid', 'abc123', {
  httpOnly: true,   // JavaScript cannot read this cookie
  secure: true,     // only sent over HTTPS
  sameSite: 'lax',  // restricts cross-site sending
  maxAge: 3600000,  // 1 hour
});

httpOnly is the flag that matters most. It hides the cookie from all JavaScript on the page, including malicious scripts injected through XSS attacks. The cookie only travels in HTTP headers, invisible to document.cookie.

Cookies are not an authentication method. They are an envelope. The question is what you put inside.

Sessions: the server remembers you

Session auth works like a valet parking. You walk in, hand over your car keys, get a numbered ticket. The cR KEYS stays with them. When you show the ticket, they find your car. When you leave, you hand back the ticket and they forget about you.

Replace "car key" with "your user data" and "ticket" with "session ID."

In Express:

const session = require('express-session');

app.use(session({
  secret: process.env.SESSION_SECRET,
  resave: false,
  saveUninitialized: false,
  cookie: {
    httpOnly: true,
    secure: process.env.NODE_ENV === 'production',
    sameSite: 'lax',
    maxAge: 24 * 60 * 60 * 1000,
  },
}));

app.post('/login', async (req, res) => {
  const user = await User.findOne({ email: req.body.email }).select('+password');
  if (!user) return res.status(401).json({ error: 'Invalid credentials' });

  const match = await user.comparePassword(req.body.password);
  if (!match) return res.status(401).json({ error: 'Invalid credentials' });

  req.session.userId = user._id;
  req.session.role = user.role;
  res.json({ name: user.name, email: user.email });
});

app.post('/logout', (req, res) => {
  req.session.destroy(() => {
    res.clearCookie('connect.sid');
    res.json({ message: 'Logged out' });
  });
});

The browser never sees or stores user data. It holds a meaningless session ID string. The server holds everything else. And because the server holds it, the server can delete it at any time. User changes their password? Destroy the session. Suspicious login? Destroy it. That instant revocation is the main thing sessions give you.

JWTs

JWT (JSON Web Token) works differently. Instead of keeping your data and giving you a reference to it, the server writes your data into a token, signs it with a secret key, and hands it to you. On every request, you show the token. The server verifies the signature, reads the data out of the token, and never looks anything up.

Think of it as a stamped letter of introduction. Someone writes "This is Priya, she is a customer, valid until 3pm," stamps it with an official seal, and gives it to her. Anyone who recognizes the seal trusts the letter without calling anyone.

A JWT has three parts separated by dots: header, payload, signature.

// Payload (base64 decoded, readable by anyone)
{
  "userId": 42,
  "role": "customer",
  "exp": 1735086400   // expiry timestamp
}

The payload is not encrypted. Anyone can decode it. The signature does not hide the data. It prevents tampering. If someone changes role from "customer" to "admin", the signature will not match and the server rejects the token. Do not put passwords or sensitive data in a JWT. It is signed, not sealed.

In Express:

import jwt from 'jsonwebtoken';

function generateToken(user) {
  return jwt.sign(
    { userId: user._id, role: user.role },
    process.env.JWT_SECRET,
    { expiresIn: '1h' }
  );
}

app.post('/login', async (req, res) => {
  const user = await User.findOne({ email: req.body.email }).select('+password');
  if (!user) return res.status(401).json({ error: 'Invalid credentials' });

  const match = await user.comparePassword(req.body.password);
  if (!match) return res.status(401).json({ error: 'Invalid credentials' });

  res.json({ token: generateToken(user) });
});

function requireAuth(req, res, next) {
  const header = req.headers.authorization;
  if (!header || !header.startsWith('Bearer ')) {
    return res.status(401).json({ error: 'No token' });
  }
  try {
    req.user = jwt.verify(header.split(' ')[1], process.env.JWT_SECRET);
    next();
  } catch {
    return res.status(401).json({ error: 'Invalid or expired token' });
  }
}

No session store. No database lookup for authentication. The server reads user data straight from the token.

The catch: once a JWT is issued, the server cannot take it back. There is no session to delete. The token lives until it expires. If you need to ban a user instantly, you either maintain a blacklist of revoked tokens (which re-introduces server-side state) or you accept a window where the token is still valid.

When to use which

Three questions I ask on every new project:

Is the client a browser talking to one backend? Sessions. Less code, instant revocation, express-session handles it.

Do multiple services need to verify identity independently? JWTs. No shared state between services.

Is instant revocation non-negotiable? Sessions. JWT blacklisting works but you lose the stateless advantage.

For the food delivery app in this series, sessions win. One backend, browser clients, and the ability to kill sessions immediately when a password changes. If I were building an API consumed by a mobile app and three microservices, JWTs.

Mistakes to avoid regardless of which you pick

Use HTTPS in production. Without it, anyone on the same wifi can copy your cookie or token in transit. Free certificates from Let's Encrypt have existed for years.

Hash passwords with bcrypt.

If using JWTs in a browser, put them in httpOnly cookies, not localStorage. Most tutorials show localStorage because it is fewer lines. It is also readable by any JavaScript on the page, including injected scripts.

Keep JWT payloads small. User ID and role. Not the full user profile.

Do not put secrets in your Git repo. Use .env files and add them to .gitignore.

References

• express-session on npm

• jsonwebtoken on npm

• MDN: HTTP cookies

• JWT.io: decode and inspect tokens

• OWASP: Session Management Cheat Sheet

• Let's Encrypt: free HTTPS certificates

Next: middleware. What it actually is, how Express processes requests through a chain, and why the order of app.use() calls matters more than you think.

console.log("1. start");

setTimeout(() => {
  console.log("2. inside timeout");
}, 0);

console.log("3. end");

I expected it to print 1, then 2, then 3. The timeout was zero milliseconds. Zero. How could anything come before it?

Output:

1. start
3. end
2. inside timeout

This confused me, I even checked if I had written the numbers wrong. The thing is, JavaScript is not broken here. This is exactly how it's supposed to work, and once you understand why, a huge chunk of async confusion just disappears.

One thing at a time for One thread

JavaScript runs on a single thread. There's one worker, and it can only do one job at a time. There is no parallel execution. It moves to next after it completes the current task.

This creates an obvious problem. If your code makes a network request that takes 3 seconds, does the entire program freeze for 3 seconds? In a traditional blocking model, yes. Everything stops, waits, then continues. That's fine for scripts but terrible for a web server that has to handle hundreds of requests.

Node.js solves this without adding more threads. The event loop is how it does that.

Think of it like a restaurant with one chef. The chef can't cook two dishes at the same time but they're not just standing at the stove staring at the pasta either. While the pasta boils, they chop vegetables, prep the sauce, plate a different dish. When the timer goes off, they come back to the pasta. One person, many things in progress, nothing truly blocked.

The event loop is the system that makes this coordination possible.

What's actually in memory when your code runs

Before getting into how the event loop works, it helps to know what's happening structurally when JavaScript executes your code.

There are two places that matter: the call stack and the task queue.

The call stack is where your code runs. When you call a function, it gets pushed onto the stack. When it returns, it pops off. JavaScript executes whatever is at the top of the stack.

The task queue is a waiting area. When an async operation finishes, like a setTimeout triggering or a file read completing, the callback gets placed in this queue.

The event loop's job is one loop that repeated forever or can say it's a while true loop which check if the call stack is empty. If it is, take the first task from the queue and push it onto the stack. That's the whole mechanism.

This is why the timeout in the example above printed last even though it was zero milliseconds. The setTimeout callback didn't run immediately. It went into the queue. The event loop didn't touch the queue until console.log("3. end") finished and the stack became empty.

Walking through the code, step by step

Let's trace exactly what happens with this code:

console.log("1. start");

setTimeout(() => {
  console.log("2. inside timeout");
}, 0);

console.log("3. end");

Step 1: console.log("1. start") gets pushed onto the call stack and runs. Prints 1. start. Pops off.

Step 2: setTimeout(...) gets pushed onto the call stack. Node hands the timer off to the browser/OS (this part happens outside the JS engine), and the callback is registered. setTimeout itself pops off immediately. The JS engine didn't wait at all.

Step 3: console.log("3. end") pushes onto the stack, runs, prints 3. end, pops off.

Step 4: The call stack is now empty. The event loop checks the task queue. The 0ms timer already fired, so the callback is sitting there waiting. The event loop pushes it onto the now-empty stack.

Step 5: console.log("2. inside timeout") runs. Prints 2. inside timeout.

Timeline:

[start]   stack: [log "1"]          -> prints "1. start"
          stack: [setTimeout]       -> hands off to Node internals, pops immediately
          stack: [log "3"]          -> prints "3. end"
          stack: []                 -> EVENT LOOP sees queue has the cb
          stack: [cb -> log "2"]    -> prints "2. inside timeout"
[end]

The timeout didn't delay by 0 milliseconds. It delayed until the stack was clear. Zero is the minimum wait, not a guarantee of immediacy.

Why Node.js specifically needs this

Node was designed for servers. A server's entire job is handling requests, and requests involve waiting: reading from databases, fetching from APIs, reading files from disk. All of that is slow relative to CPU speed.

If Node ran synchronously, a single slow database query would freeze the server. Every other user would sit and wait. You'd need one thread per user, which is how older servers like Apache worked. That works until it doesn't, and it falls over somewhere in the hundreds or low thousands of concurrent connections because threads are expensive.

Node's event loop lets a single thread handle thousands of connections. While it's waiting on a database response for user A, it's processing user B's request. When the database responds for A, that callback goes in the queue and gets handled when the stack clears. No thread spawning, no context switching overhead.

This is why Node became popular for APIs and real-time apps because this model handles I/O-heavy workloads with very low resource usage.

Async operations: where do they actually go

When you call setTimeout, fs.readFile, fetch, or pretty much anything async, you're handing work to something outside the JS engine. Node.js has a layer called libuv that handles these operations using the operating system's async capabilities and a thread pool for things that aren't natively async (like some file operations).

Your JS Code
     |
     v
Node.js Runtime
     |
     ----> V8 Engine (executes JS synchronously)
     |
     ----> libuv (handles async I/O, timers, etc.)
                |
                ----> OS networking (epoll/kqueue/IOCP)
                ----> Thread pool (file I/O, crypto, etc.)

When an async operation completes in libuv, the callback gets handed to the event loop, which puts it in the appropriate queue. Then the event loop picks it up when the call stack is empty and runs it.

This is why you can write code like this and it works fine:

const fs = require('fs');

fs.readFile('bigfile.txt', 'utf8', (err, data) => {
  console.log('file done');
});

console.log('this runs while file is being read');

Output:

this runs while file is being read
file done

The readFile call dispatched the work and returned immediately. Your next line ran. Somewhere later, the file read finished, the callback arrived in the queue, and the event loop ran it once the stack was free.

Timers vs I/O callbacks

Timers (setTimeout, setInterval) and I/O callbacks don't go into the same queue. The event loop has multiple phases, and it visits them in order. Timers fire in one phase, I/O callbacks fire in another, and there are special microtask queues (for Promises) that run between phases.

The simplified picture:

Event Loop Phases (simplified):

  [timers]         -> setTimeout, setInterval callbacks
       |
  [I/O callbacks]  -> file reads, network, etc.
       |
  [poll]           -> wait for new I/O if nothing pending
       |
  [check]          -> setImmediate callbacks
       |
  [close]          -> cleanup callbacks
       |
  loop back to [timers]

Between EVERY phase:
  [microtask queue] -> Promise .then(), queueMicrotask()
  (runs to completion before next phase starts)

The microtask queue (Promises) is the sneaky one. It runs to completion before the event loop moves to the next phase. So Promise callbacks have higher priority than setTimeout callbacks.

This is why:

console.log("start");

setTimeout(() => console.log("setTimeout"), 0);

Promise.resolve().then(() => console.log("promise"));

console.log("end");

Output:

start
end
promise
setTimeout

The Promise resolved synchronously but its .then() callback went into the microtask queue. The setTimeout callback went into the timer queue. After console.log("end") cleared the stack, the event loop drained the microtask queue first, then moved to timers.

What "non-blocking" actually means in practice

You'll see "non-blocking I/O" used to describe Node everywhere. When you do I/O (reading a file, making a network call), Node doesn't block the call stack waiting for it. It hands the work off and continues. The callback runs later.

The implication is that your JavaScript code is always running, never waiting. If something seems to be blocking, it's almost always CPU-heavy synchronous JavaScript rather than I/O. A function that does massive string manipulation or a tight loop will actually block Node because that work stays on the call stack.

// this blocks Node completely while running
function blockForMs(ms) {
  const end = Date.now() + ms;
  while (Date.now() < end) {}
}

setTimeout(() => console.log("timer"), 100);

blockForMs(3000); // blocks for 3 seconds

// the timer callback fires at 3000ms+, not 100ms

The timer was set for 100ms but the while loop held the call stack hostage for 3 seconds. The event loop couldn't check the queue. This is called "blocking the event loop" and it's the thing Node developers actively try to avoid.

For CPU-heavy work, you'd use worker threads or break the work into smaller chunks. But for I/O, which is most of what servers do, the event loop handles it cleanly.

A slightly more realistic example

Here's something closer to actual server code:

const fs = require('fs');

console.log("server starting...");

// imagine these are three incoming requests hitting at the same time
fs.readFile('user1.json', 'utf8', (err, data) => {
  console.log('user1 data ready');
});

fs.readFile('user2.json', 'utf8', (err, data) => {
  console.log('user2 data ready');
});

setTimeout(() => {
  console.log('cleanup timer fired');
}, 50);

console.log("all requests dispatched, waiting...");

Output (order of user1/user2 depends on which file read finishes first):

server starting...
all requests dispatched, waiting...
user1 data ready
user2 data ready
cleanup timer fired

Three async operations dispatched in rapid succession. None of them blocked the others. The last console.log ran before any of them returned because the stack clears synchronous code first. Then the event loop started processing callbacks as they arrived.

Scalability: why this matters for real apps

The event loop model is what lets Node handle high concurrency without throwing more hardware at the problem. A traditional server doing 1000 concurrent requests would have 1000 threads, each consuming memory and context switching time. Node handles those same 1000 requests in a single thread, with 1000 callbacks sitting in queues, processed one at a time as responses come in.

There are tradeoffs. CPU-intensive tasks hurt Node because they block the loop. Long synchronous operations hurt because they starve everything else in the queue. The model rewards I/O-heavy code that spends most of its time waiting, which is exactly what most APIs do.

Real apps at scale also use Node's cluster module or run multiple Node processes behind a load balancer to take advantage of multiple CPU cores. The event loop being single-threaded doesn't mean you're limited to one core. It means each Node process is single-threaded.

The call stack runs your synchronous code. One thing at a time, top to bottom. When you call an async function, the actual work is handed off outside the JS engine, and a callback is registered. When that work completes, the callback goes into a queue. The event loop watches the call stack, and when it's empty, it pulls from the queue and runs the next callback.

Microtasks (Promises) run before timers. Timers run before I/O callbacks in some phases. But for day-to-day code, the thing that matters is: synchronous code runs first, then queued callbacks, and Promises resolve before setTimeout.

You don't need to memorise the phase order to write good async code. You need to understand that the event loop is a queue manager, that your synchronous code always runs before any callbacks, and that blocking the call stack with heavy synchronous work kills your app's responsiveness.

References

• Node.js docs: The Node.js Event Loop : the official explanation, worth reading once you're comfortable with the basics

• Philip Roberts: What the heck is the event loop anyway? : the talk that explained this to JS developers, still the best visual walkthrough

• Jake Archibald: Tasks, microtasks, queues and schedules : the definitive deep dive on microtasks vs task queues

• libuv documentation : if you want to understand the layer below Node that actually does the async I/O

MongoDB is not always the right choice, but for a JavaScript developer learning backend for the first time, the mental model clicks in a way that SQL tables do not. Your data looks like objects. Your queries feel like JavaScript. You do not have to learn an entirely separate language just to get something saved and retrieved. You write code that looks like this:

const user = {
  name: "Abc",
  email: "abc@example.com",
  age: 28
}

And that is also, more or less, exactly what gets stored. There is no translation layer in your head where you have to think "okay, this object maps to these columns in this table with these foreign keys." It is just data, shaped the way you already think about data in JavaScript.

That familiarity is MongoDB's biggest advantage when you are starting out. It gets out of your way.

What is MongoDB

MongoDB is a database, but it stores data differently from databases like MySQL or PostgreSQL. Those are relational databases that store data in tables with rows and columns, like a spreadsheet. MongoDB is a document database. It stores data as documents, which are essentially JSON objects, and groups them into collections instead of tables.

So instead of a users table where every row has the same columns, you have a users collection where each document can have a slightly different shape. One user might have a phone field, another might not. One might have a nested address object, another might not. MongoDB simply save the data into the fields as it receives.

Each document automatically gets a unique _id field. MongoDB generates this as an ObjectId, unique across the entire database. _id is used to fetch specific documents.

The flexibility is real when your application is young and you are still figuring out what data you actually need. You can add a field to some documents without running a migration script. You can change your mind about the shape of your data without a painful database alteration. When you are building version one of something and requirements are shifting every week, this matters.

That said, the flexibility has a cost. Nothing stops two developers on your team from storing the same concept under different field names. Nothing stops your code from saving a string where a number was expected. MongoDB will happily store anything, and you find out something was wrong not when you save the data, but when you try to read it three weeks later and it comes back in an unexpected shape.

This problem gets solved by Mongoose.

Why you cannot just use MongoDB directly in your Node app

Technically, you can. MongoDB has an official Node.js driver that lets you connect and run queries without any extra library. But the raw driver gives you back plain JavaScript objects with no type information. Your editor cannot autocomplete field names. If you make a typo, nothing warns you. If someone saves { eMail: "..." } instead of { email: "..." }, the database accepts it without complaint.

For a small personal project, that is probably fine. For anything with more than one developer, or anything you want to maintain six months from now, you start wanting structure.

Mongoose is a library that sits between your Node.js code and MongoDB. It is often called an ODM (Object Document Mapper), which is the document database equivalent of an ORM. You define the shape of your data in code, and the library makes sure anything going to the database matches that shape. You get schemas, validation, methods on documents, hooks that run before or after operations, and a proper query API.

Setting up the connection

Before any schema work, you connect. Mongoose wraps MongoDB's connection handling so you do it once and every model you define automatically uses that connection.

// db.js
import mongoose from 'mongoose';

async function connectDB() {
  try {
    await mongoose.connect(process.env.MONGO_URI);
    console.log('Connected to MongoDB');
  } catch (err) {
    console.error('MongoDB connection failed:', err);
    process.exit(1);
  }
}

export default connectDB;

// server.js
import 'dotenv/config';
import connectDB from './db.js';

async function startServer(){
    await connectDB();
}

The connection string points to your MongoDB instance and names the database. For a local setup it looks like mongodb://localhost:27017/foodapp. For MongoDB Atlas, it comes from the dashboard and works identically. Always store it in a .env file. Hardcoding it is fine while learning but you should not commit credentials to Git.

Writing a schema

A Mongoose schema is where you tell your application what a document is supposed to look like. You create one with new mongoose.Schema({}), passing an object where each key is a field name and the value describes the expected type.

We will build a food delivery app through this post. The user schema starts simple:

// user/user.model.js
import { Schema, model } from 'mongoose';

const userSchema = new Schema({
  name:     String,
  email:    String,
  phone:    String,
  age:      Number,
  role:     String,
  isActive: Boolean,
});

export const User = model('User', userSchema);

The model() call is what turns a schema into something you can actually use. The name you pass, 'User', determines the MongoDB collection name. Mongoose lowercases and pluralizes it automatically, so documents land in a collection called users.

Using the model looks exactly like working with a class:

import { User } from './models/User.js';

async function createUser() {
  const user = new User({
    name:  'John Doe',
    email: 'john@example.com',
    phone: '9999999999',
    age:   25,
  });

  await user.save();
  console.log(user._id); // MongoDB auto-generates a unique ID
}

createUser();

This works, but the schema above does not enforce much. You can save a user with no email, an age of -500, or a role called 'supervillain'. Mongoose lets it all through because you only told it the type, not the rules. For that, you need constraints.

Adding constraints

Constraints are what make a schema actually useful. Instead of just passing a type, you pass an object with the type and whatever rules you want enforced.

const userSchema = new Schema({
  name: {
    type:      String,
    required:  [true, 'Name is required'],
    trim:      true,
    minlength: [2,  'Name must be at least 2 characters'],
    maxlength: [60, 'Name cannot exceed 60 characters'],
  },
  email: {
    type:      String,
    required:  [true, 'Email is required'],
    unique:    true,
    lowercase: true,
    trim:      true,
    match:     [/^\S+@\S+\.\S+$/, 'Please enter a valid email'],
  },
  phone: {
    type:  String,
    match: [/^[6-9]\d{9}$/, 'Enter a valid 10-digit mobile number'], // Regex for Indian phone no.
  },
  age: {
    type: Number,
    min:  [16, 'Must be at least 16 to order'],
    max:  [120, 'Please enter a valid age'],
  },
  role: {
    type:    String,
    enum:    ['customer', 'rider', 'admin'],
    default: 'customer',
  },
  isActive: {
    type:    Boolean,
    default: true,
  },
}, {
  timestamps: true,
});

The timestamps: true option automatically adds createdAt and updatedAt fields to every document and keeps updatedAt current on every save, without you having to remember to do it manually.

When validation fails, Mongoose throws a ValidationError before the document reaches MongoDB. The error includes your custom message, so you can send it directly back to the user:

try {
  const bad = new User({ name: 'A', email: 'notanemail' });
  await bad.save();
} catch (err) {
  console.log(err.errors.name.message);
  // "Name must be at least 2 characters"
  console.log(err.errors.email.message);
  // "Please enter a valid email"
}

One thing worth knowing about unique: true: it creates a database level index in MongoDB rather than running a Mongoose check. The error looks different, instead of a ValidationError you get an error with err.code === 11000.

Sometimes the built-in options are not enough. You can pass a custom validator function for any field:

deliveryPincode: {
  type: String,
  validate: {
    validator: function(v) {
      return /^\d{6}$/.test(v); 
    },
    message: (props) => `${props.value} is not a valid pincode`,
  },
},

Nested objects and references

Data in real applications is never straightforward. A user has an address. An order has multiple items. Mongoose gives you two ways to handle this: embed the related data directly inside the document, or store a reference to another document by its _id and look it up separately.

For an address, embedding makes sense. You would never want to fetch an address without the user it belongs to, and an address has no meaning outside of that context. The schema looks like a nested object:

address: {
  street:  { type: String, trim: true },
  city:    { type: String, trim: true },
  pincode: {
    type:  String,
    match: [/^\d{6}$/, 'Invalid pincode'],
  },
},

You read it by chaining dots: user.address.city.

For an order that belongs to a user, a reference makes more sense. A user exists independently and appears in many places. You store the user's _id in the order:

// models/order.model.js
const orderSchema = new Schema({
  user: {
    type: mongoose.Schema.Types.ObjectId,
    ref:  'User',
    required: true,
  },
  restaurant: {
    type: mongoose.Schema.Types.ObjectId,
    ref:  'Restaurant',
    required: true,
  },
  items: [{
    name:     { type: String, required: true },
    quantity: { type: Number, min: 1, default: 1 },
    price:    { type: Number, required: true, min: 0 },
  }],
  totalAmount: { type: Number, required: true },
  status: {
    type:    String,
    enum:    ['placed', 'confirmed', 'out_for_delivery', 'delivered', 'cancelled'],
    default: 'placed',
  },
}, { timestamps: true });

The ref: 'User' tells Mongoose which model to look up when you call .populate():

const order = await Order
  .findById(orderId)
  .populate('user', 'name email')
  .populate('restaurant', 'name address');

console.log(order.user.name); // Prints the name of the user of the particular id

Without populate(), order.user is just an ID string. With it, Mongoose runs the extra lookup and replaces the ID with the actual document. The rule for choosing is simpler than it sounds: if the data only makes sense alongside its parent and will always be read together with it, embed it. If the data exists on its own and gets used from multiple places, reference it.

Virtual fields

A virtual is a field that does not get saved to the database. It is computed from other fields whenever you read the document. For the food delivery app, a displayAddress virtual combines the separate address fields into one readable string:

userSchema.virtual('displayAddress').get(function() {
  if (!this.address?.city) return 'No address saved';
  return `\({this.address.street}, \){this.address.city} - ${this.address.pincode}`;
});

const user = await User.findOne({ email: 'john@example.com' });
console.log(user.displayAddress);
// example:"Civil Lines, Delhi - 110054"

Virtuals do not appear in JSON.stringify() output by default, which means they will not show up in API responses unless you enable them. Add toJSON: { virtuals: true } to the schema options:

const userSchema = new Schema({ /* fields */ }, {
  timestamps: true,
  toJSON:   { virtuals: true },
  toObject: { virtuals: true },
});

Use regular functions, not arrow functions, for virtuals and hooks. Arrow functions do not have their own this, so this.address inside an arrow function refers to nothing useful. Mongoose passes the document as this, and a regular function is what receives it correctly.

Hooks

A hook is a function that runs automatically before or after a database operation.

The most important hook in almost every app is hashing a password before saving it. You should never store plain text passwords in a database:

import bcrypt from 'bcryptjs';

// Mongoose handles the promise through async; hence next() is not required
// if async absent, use next()
userSchema.pre('save', async function() {
  
  if (!this.isModified('password')) return;

  this.password = await bcrypt.hash(this.password, 12);
});
// 12 is Cost Factor; Number of iterations for hashing

The isModified('password') check is necessary. Without it, every time you update any field on a user, the hook would rehash an already hashed password. Checking isModified means the hook only runs when the password field actually changed.

Post hooks run after the operation completes:

// send welcome email to new users
userSchema.post('save', async function (doc) {
  if (doc._wasNew) {
    try {
      await sendWelcomeEmail(doc.email, doc.name);
    } catch (err) {
      console.error("Email failed to send:", err);
    }
  }
});

There is a gotcha with update operations. When you use findByIdAndUpdate(), the pre('save') hook does not fire. Mongoose treats updates differently from saves. If you want validation to run on updates too:

userSchema.pre('findOneAndUpdate', function() {
  this.setOptions({ runValidators: true });
});

Instance methods and static methods

Instance methods run on individual documents. Static methods run on the model class itself.

The most common instance method is password comparison for login:

userSchema.methods.comparePassword = async function(plainTextPassword) {
  return bcrypt.compare(plainTextPassword, this.password);
};

// in your login route:
const user = await User.findOne({ email: req.body.email }).select('+password');
const match = await user.comparePassword(req.body.password);
if (!match) return res.status(401).json({ error: 'Wrong password' });

Use .select('+password') when you mark a field with select: false in the schema, Mongoose leaves it out of all query results by default. The +password syntax opts it back in for this one query where you actually need it.

Static methods are good for reusable query logic that belongs on the model itself:

userSchema.statics.findActiveRiders = async function(city) {
  return this.find({
    role: 'rider',
    isActive: true,
    'address.city': city,
  });
};

const riders = await User.findActiveRiders('Mumbai');

Querying

Mongoose wraps MongoDB's query system in a clean API. These operations cover almost everything you will need day to day:

// all customers
const users = await User.find({ role: 'customer' });

// find one by email
const user = await User.findOne({ email: 'john@example.com' });

// find by MongoDB _id
const user = await User.findById(userId);

// update and get back the updated version
const updated = await User.findByIdAndUpdate(
  userId,
  { isActive: false },
  { new: true } // this returns the document after change
);

// delete
await User.findByIdAndDelete(userId);

// pick specific fields, sort, paginate
const page2 = await User
  .find({ role: 'customer' })
  .select('name email')
  .sort({ createdAt: -1 })
  .skip(10)
  .limit(10);

.select() is worth understanding early. Controlling which fields come back is how you avoid accidentally sending password hashes to the frontend.

Indexes

Indexes were one concept that took me a while to wrap my head around. Without indexes, MongoDB reads every document in a collection one by one to find a match. That is fine for a hundred documents. At a hundred thousand, a query that took 2ms starts taking seconds. The culprit is almost always a missing index.

An index is a separate, sorted data structure that MongoDB maintains alongside your collection. When you query a field that has an index, MongoDB looks up the value in that structure instead of scanning every document. The difference in speed can be several orders of magnitude.

You add indexes inside the schema in Mongoose. unique: true on the email field already creates one automatically. For other fields, you either add index: true on the field definition or call schema.index() directly:

// option 1: inline on the field
email: { type: String, unique: true }, // index created automatically
role:  { type: String, index: true  }, // single-field index

// option 2: schema.index() for more control
userSchema.index({ role: 1, isActive: 1 });    // compound index
userSchema.index({ 'address.city': 1 });       // index on a nested field
userSchema.index({ createdAt: -1 });           // -1 = descending order

The 1 means ascending order, -1 means descending. For most lookups the direction does not matter much, but for sorted queries it can. A compound index covers queries that filter by multiple fields together. If your app frequently fetches active riders in a specific city, an index on { role: 1, isActive: 1, 'address.city': 1 } will serve that query far faster than three separate single-field indexes.

There is a tradeoff. Indexes speed up find() but slightly slow down save(), because MongoDB has to update every index whenever a document changes. For most applications this is completely fine. The read speedup far outweighs the write cost. But indexing every field by default is not a good habit. Add indexes for fields you actually filter or sort by. Also, if you're adding an index to a live database with millions of records, do it during off-peak hours. Building an index on a massive active collection can lock the collection or spike CPU usage, leading to a temporary slowdown for your users.

A special case worth knowing is the TTL (Time To Live) index. It automatically deletes documents after a certain amount of time. This is perfect for email verification tokens, password reset links, or session records that should expire on their own:

const tokenSchema = new Schema({
  userId:    { type: mongoose.Schema.Types.ObjectId, required: true },
  token:     { type: String, required: true },
  createdAt: { type: Date, default: Date.now },
});

// MongoDB automatically deletes these documents 1 hour after createdAt
tokenSchema.index({ createdAt: 1 }, { expireAfterSeconds: 3600 });

MongoDB handles the deletion automatically without a cleanup script.

Error handling

MongoDB and Mongoose throw different types of errors, and you need to handle them differently. Bundling all errors into a single catch block and returning a 500 response is a pattern that makes debugging very frustrating.

There are three categories you will encounter regularly. The first is a ValidationError from Mongoose, which fires when a document fails schema constraint checks. The second is error code 11000 from MongoDB itself, which fires when a unique constraint is violated at the database level. The third is a CastError, which fires when you pass something to findById() that is not a valid MongoDB ObjectId format.

// a reusable error handler you can drop into any project
function handleMongooseError(err) {
  // validation failed before reaching the database
  if (err.name === 'ValidationError') {
    const messages = Object.values(err.errors).map(e => e.message);
    return { status: 400, message: messages.join(', ') };
  }

  // duplicate key: unique constraint violated
  if (err.code === 11000) {
    const field = Object.keys(err.keyValue)[0];
    return { status: 409, message: `${field} already exists` };
  }

  // invalid ObjectId format passed to findById or similar
  if (err.name === 'CastError') {
    return { status: 400, message: 'Invalid ID format' };
  }

  return { status: 500, message: 'An unexpected error occurred on the server.' };
}

If your route receives an id parameter from a URL and that parameter is not a valid ObjectId, calling User.findById(id) throws before even hitting the database. Without handling it, that becomes an unhandled exception. With it, you return a clean 400 response.

Using this in a route looks like:

app.post('/users', async (req, res) => {
  try {
    const user = await User.create(req.body);
    res.status(201).json(user);
  } catch (err) {
    const { status, message } = handleMongooseError(err);
    res.status(status).json({ error: message });
  }
});

Aggregation pipelines

Once your data is in MongoDB and you understand basic querying, you will eventually need to answer questions like: what is the total revenue per restaurant this month, how many orders has each rider completed, what is the average order value by city. Basic find() queries cannot answer these. Aggregation pipelines can.

An aggregation pipeline is a sequence of stages that transform documents. Each stage takes documents in, does something to them, and passes the result to the next stage. Think of it like an assembly line for your data.

const revenueByRestaurant = await Order.aggregate([
  // stage 1: only delivered orders from this month
  {
    $match: {
      status:    'delivered',
      createdAt: { $gte: new Date('2024-12-01') },
    }
  },
  // stage 2: group by restaurant and sum totals
  {
    $group: {
      _id:          '$restaurant',
      totalRevenue: { \(sum: '\)totalAmount' },
      orderCount:   { $sum: 1 },
    }
  },
  // stage 3: sort by revenue, highest first
  {
    $sort: { totalRevenue: -1 }
  },
  // stage 4: join with restaurants to get the name
  {
    $lookup: {
      from:         'restaurants',
      localField:   '_id',
      foreignField: '_id',
      as:           'restaurantInfo',
    }
  },
  // stage 5: reshape the output
  {
    $project: {
      restaurantName: { \(arrayElemAt: ['\)restaurantInfo.name', 0] },
      totalRevenue:   1,
      orderCount:     1,
    }
  }
]);

\(match is like a find() filter. Put it as early as possible in the pipeline so you are working with fewer documents in the stages that follow. \)group is how you aggregate: you specify a _id to group by and then use accumulator operators like \(sum, \)avg, \(min, and \)max to compute values. \(lookup is MongoDB's join equivalent. It pulls in documents from another collection and attaches them to the current documents. \)project controls which fields appear in the output, similar to .select() on a regular query.

These four stages cover the vast majority of reporting and analytics work you will do in a real application.

Transactions

By default, a MongoDB write either succeeds or fails on its own. That is fine most of the time. But sometimes you have an operation that involves multiple writes that must all succeed together or all fail together. Charging a user for an order and creating the order record is the classic example. If the charge succeeds but the order creation fails, the user paid for something that does not exist. That is a bad place to be.

Mongoose exposes them through sessions:

const session = await mongoose.startSession();

try {
  session.startTransaction();

  // both of these run inside the same transaction
  const order = await Order.create([{
    user:        userId,
    restaurant:  restaurantId,
    items:       cartItems,
    totalAmount: total,
    status:      'placed',
  }], { session });

  await User.findByIdAndUpdate(
    userId,
    { $inc: { orderCount: 1 } },
    { session }
  );

  await session.commitTransaction();
  return order[0];

} catch (err) {
  // if anything fails, roll back both writes
  await session.abortTransaction();
  throw err;

} finally {
  session.endSession();
}

The key is passing { session } to every operation inside the transaction. Without it, that operation runs outside the transaction and will not be rolled back if something fails. The finally block ensures the session is always closed, even when an error is thrown.

The complete schema

Pulling it all together, here is the full User model with constraints, a nested address, a virtual, the password hook, the login method, and indexes:

// models/user.model.js
import { Schema, model } from 'mongoose';
import bcrypt from 'bcryptjs';

const userSchema = new Schema({
  name: {
    type:      String,
    required:  [true, 'Name is required'],
    trim:      true,
    minlength: [2,  'Name must be at least 2 characters'],
    maxlength: [60, 'Name cannot exceed 60 characters'],
  },
  email: {
    type:      String,
    required:  [true, 'Email is required'],
    unique:    true,
    lowercase: true,
    trim:      true,
    match:     [/^\S+@\S+\.\S+$/, 'Please enter a valid email'],
  },
  password: {
    type:      String,
    required:  [true, 'Password is required'],
    minlength: [8, 'Password must be at least 8 characters'],
    select:    false,
  },
  phone: {
    type:  String,
    match: [/^[6-9]\d{9}$/, 'Enter a valid 10-digit mobile number'],
  },
  address: {
    street:  { type: String, trim: true },
    city:    { type: String, trim: true },
    pincode: { type: String, match: [/^\d{6}$/, 'Invalid pincode'] },
  },
  role: {
    type:    String,
    enum:    ['customer', 'rider', 'admin'],
    default: 'customer',
  },
  isActive: { type: Boolean, default: true },
}, {
  timestamps: true,
  toJSON:   { virtuals: true },
  toObject: { virtuals: true },
});

// indexes
userSchema.index({ role: 1, isActive: 1 });
userSchema.index({ 'address.city': 1 });

// virtual
userSchema.virtual('displayAddress').get(function() {
  if (!this.address?.city) return 'No address saved';
  return `\({this.address.street}, \){this.address.city} - ${this.address.pincode}`;
});

// hooks
userSchema.pre('save', async function() {
  if (!this.isModified('password')) return;
  this.password = await bcrypt.hash(this.password, 12);
});

userSchema.pre('findOneAndUpdate', function() {
  this.setOptions({ runValidators: true });
});

// instance method
userSchema.methods.comparePassword = async function(candidate) {
  return bcrypt.compare(candidate, this.password);
};

export const User = model('User', userSchema);

How it all fits together

It helps to see the full picture in one place. Your application code talks to Mongoose. Mongoose talks to MongoDB. The schema sits in the middle describing what is allowed, what gets validated, what runs automatically, and how queries behave.

Your route calls a method on the model. The model checks the schema constraints. Any matching pre-hooks run. The data goes to MongoDB. Post-hooks run. The result comes back as a typed Mongoose document with your virtual fields and methods attached.

MongoDB is a genuinely good place to start. The document model matches how JavaScript developers already think about data, the queries are readable, and you can get something working quickly without a lot of infrastructure. Mongoose adds the structure that MongoDB lacks on its own, which is the right tradeoff for most Node.js projects.

References

• Mongoose documentation: Schemas

• Mongoose documentation: Validation

• Mongoose documentation: Middleware

• Mongoose documentation: Populate

• Mongoose documentation: Virtuals

• Mongoose documentation: Indexes

• Mongoose documentation: Transactions

• MongoDB docs: Aggregation pipeline operators

• MongoDB docs: Indexes

• MongoDB Atlas

• bcryptjs on npm: password hashing library

People told me Linux had a learning curve. Except this curve was a cliff.

If you want a 12-minute brush through Linux basics before diving into the weird stuff, Fireship made a video for exactly that.

Before we go into a rabbit hole, I will suggest to try all of this on a VPS, a VM, or a Docker container. Not your main machine. The reason is simple: some of what I will show you involves deliberately breaking things. Try to understand the command before copying, because imagine copying rm -rf /. But if you are HIM, nobody is stopping you. On a throwaway container? You type exit, spin up a fresh one, and try again. Freedom to destroy without consequences is genuinely how you learn Linux fastest.

Two tools that will save you more than any tutorial

Before anything else: man and tldr. These are your cheat codes.

man is the built-in Linux manual. Every command, every system file, every config format has one. When you wonder what a flag does or what a file is for, man has the answer written by the people who built the thing.

man ls
man chmod
man 5 passwd      # the 5 means file format manual, not the command
man resolv.conf   # the config file itself has a man page

The problem with man is density. It was written by engineers for engineers. Enter tldr, a community-written alternative that gives you just the practical common cases.

apt install tldr
tldr ls
tldr chmod

tldr gives you five useful examples instead of fifty pages of specification. Use tldr when you want to quickly understand what something does. Use man when you need the full picture. If man is the textbook, tldr is the cheat sheet your classmate scribbled the night before the exam.

One more habit worth building early: when something fails, read the actual error message before googling. "Permission denied" means permissions. "No such file or directory" means the path is wrong or the file doesn't exist. "Command not found" means it is not installed or not in your PATH. Linux error messages are not trying to confuse you. They are telling you what happened.

What even is /etc

/etc is where Linux keeps its brain. Configuration for almost everything the system does lives here in plain text files you can open with any editor. I spent the first hour just running ls /etc and opening random files, and breaking things up. That felt like finding an unlocked door in a building I thought was sealed.

DNS redirection or how I broke Google with one line

The one thing I tried in /etc was /etc/hosts. I wanted to reach my local dev server by a name instead of 127.0.0.1:3000. Opening the file:

nano /etc/hosts

127.0.0.1   localhost
::1         localhost ip6-localhost

I added one line at the bottom and saved:

127.0.0.1   myapp.local

It worked immediately. No restart, no daemon reload. The file is read fresh on every single lookup because this check happens before Linux even thinks about asking a DNS server anything.

Then I had an idea. What if I pointed instagram.com at localhost?

nano /etc/hosts
# add: 127.0.0.1   instagram.com

ping instagram.com

PING instagram.com (127.0.0.1) 56 bytes of data.

This is poor man's parental control. All of Meta's infrastructure, the entire DNS system, bypassed by one line in a text file on your machine. The whole internet is the fallback. This file runs first.

Imagine making your friend slowly lose their mind on a shared Linux machine, redirect their most visited sites to 127.0.0.1 in /etc/hosts. They will restart their wifi three times, blame their ISP, factory reset their router, and spend an hour in a chat with customer support before anyone thinks to check a text file.

/etc/resolv.conf: the file that keeps rewriting itself

Once I understood /etc/hosts, I wanted to know what happened when a hostname was not found there. That fallback is /etc/resolv.conf, which holds the actual DNS server address the system asks.

cat /etc/resolv.conf

# This file is managed by man:systemd-resolved(8). Do not edit.
nameserver 127.0.0.53

I had edited this on my machine and my changes kept vanishing. Now I understood why. systemd-resolved is a local DNS middleman running at 127.0.0.53. It manages this file and rewrites it whenever it restarts.

Let's break the internet:

echo "nameserver 0.0.0.0" > /etc/resolv.conf
ping google.com

ping: google.com: Temporary failure in name resolution

The machine is still connected to the network. It just cannot resolve any names because it is asking a DNS server that does not exist. apt update fails, curl fails, package installs fail. The network is fine, only name resolution is dead. This is what a misconfigured DNS server looks like from the inside.

# Fix it immediately
echo "nameserver 8.8.8.8" > /etc/resolv.conf

The permanent fix without the file fighting back is editing systemd-resolved's own config:

nano /etc/systemd/resolved.conf
# under [Resolve]:
# DNS=1.1.1.1
# FallbackDNS=8.8.8.8
systemctl restart systemd-resolved

/etc/passwd and /etc/shadow: the user accounts are just a text file

I heard that everything in Linux is treated as a file, from hardware device to system configurations, just a file. This in itself was surprising to me, till I started exploring it through /etc/passwd

cat /etc/passwd

root:x:0:0:root:/root:/bin/bash
daemon:x:1:1:daemon:/usr/sbin:/usr/sbin/nologin
www-data:x:33:33:www-data:/var/www:/usr/sbin/nologin
saumya:x:1001:1001::/home/saumya:/bin/bash

# the above is in the form of name:password:UID:GID:GECOS:directory:shell (root:x:0:0:root:/root:/bin/bash)

Seven colon-separated fields per line: username, password placeholder, user ID, group ID, comment, home directory, login shell. The x in the second field means the actual password hash is stored in /etc/shadow, which only root can read.

sudo cat /etc/shadow | grep saumya

saumya:\(6\)rounds=4096\(saltstring\)hashedpassword:19820:0:99999:7:::

# 9 fields separated by ':'

The 9 fields are username, encrypted password, last change, minimum age, maximum age, warning, activity, expiration and reserved.

\(6\) means SHA-512. The rounds=4096 deliberately makes hashing slow. Slow hashing means brute-force attacks take longer. Your password is protected partly by intentional computational expense.

Now the interesting part. Every account with /usr/sbin/nologin as its login shell cannot open an interactive session. Try to SSH in as www-data and the connection closes immediately. No password prompt, no shell, nothing. These accounts exist only so the processes running as them have a numeric user identity for permissions. They are not meant for humans to log into.

Now, i wanted to try breaking it and hence, changed my login shell to /bin/false in /etc/passwd.

nano /etc/passwd

# find your line, change /bin/bash to /bin/false

When i tried to connect in a new terminal, the session closes the instant the shell starts because /bin/false exits immediately with a failure code. My account exists, password is correct, the SSH connection succeeds, but I am immediately disconnected. I have locked yourself out without deleting the account or changing the password.

I fixed the current session by changing /bin/false back to /bin/bash.

/etc/ssh/sshd_config: watching bots try your front door

On any internet-facing VPS, run this after it has been up for a day:

grep "Failed password" /var/log/auth.log | wc -l

On a fresh VPS after few hours, that number is usually in the thousands. Automated bots scan every IP address on the internet looking for open port 22 and trying common username and password combinations. root, admin, ubuntu, 123456. Constantly, from everywhere.

All SSH behavior is controlled by /etc/ssh/sshd_config:

nano /etc/ssh/sshd_config

You would see a file with multiple lines with Port info, permits and authentications. Two lines that matter immediately:

PasswordAuthentication no
PermitRootLogin no

Setting PasswordAuthentication no means SSH only accepts key-based authentication. The bots still knock but there is nothing to brute-force. No password means no attack surface. PermitRootLogin no means even with valid root credentials, nobody SSHs directly into root.

Want to watch the bots in real time:

tail -f /var/log/auth.log

Run that for five minutes on a fresh VPS. The attempts come in constantly from IP addresses across dozens of countries. It is mildly unsettling and also clarifying. This is just what the internet is like all the time.

After disabling passwords:

systemctl restart sshd
grep "Failed password" /var/log/auth.log | tail -5
# bots still try, still fail, but now they fail faster

Whenever the bots try a password, the server tells them 'NO', not even letting them trying the password to authenticate.

/etc/fstab: the file that can stop your machine from booting

/etc/fstab is read at boot. Linux mounts everything listed here. USB drives, additional partitions, network shares. Get an entry wrong and the machine may not boot.

cat /etc/fstab

# in the order
# file system, mount point, type, options, dump and pass


UUID=abc123   /        ext4    defaults    0 1
UUID=def456   /boot    ext4    defaults    0 2
tmpfs         /tmp     tmpfs   defaults    0 0

UUID is used instead of the device name because the name might change but the UUID won't. This acts as the Aadhar Number of the device.

The experiment:

nano /etc/fstab
# add at the bottom:
broken-entry-here

'broken-entry-here' is a string of text without the columns. When the mount command encounters it, it has no idea what the device is, and where it should be mounted.

Now test without rebooting:

mount -a

This command is essentially the dry run of the boot process.

mount: /: can't find broken-entry-here.

Error. Exactly what you would get on boot if you saved this and restarted. Fix it by removing or commenting out the broken line, then run mount -a again to confirm it passes.

The rule: always test with mount -a before rebooting when you touch /etc/fstab. Always. A bad fstab entry drops you into recovery mode or an emergency shell with no obvious explanation. Keep a mental note that this file exists and kills boots if you get it wrong.

/proc: the folder that is not actually on disk

/proc is a virtual filesystem. There are no files here stored on disk anywhere. When you read something from /proc, the kernel generates the content on the spot from its own live internal data. It is a window into running system state, not stored data.

cat /proc/meminfo

Run it twice, ten seconds apart. The numbers change. Because it is live.

cat /proc/cpuinfo | grep "flags" | head -1

The flags like vmx, aes and sse which details the CPU capability the processor supports. vmx means hardware virtualization is available. If you try to run KVM and it refuses, check for this flag. aes means the CPU has built-in AES hardware acceleration. Disk encryption and TLS are genuinely faster because of this single flag. sse/avx means CPU has Streaming SIMD Extensions which can boost performance for heavy computations, and video encoding.

The per-process data is where things get interesting. Every running process gets its own directory:

# Find something running
ps aux | grep nginx
# Say it's PID 1234

ls /proc/1234/

cmdline  cwd  environ  exe  fd  maps  mem  net  status

fd/ lists every file descriptor the process has open right now. Log files, sockets, pipes, all of it as symlinks. environ has the exact environment variables the process launched with. cmdline shows the full command that started it. status has memory use, current state, which user it runs as.

cat /proc/1234/cmdline | tr '\0' ' '
ls -la /proc/1234/fd/

The kernel stores command arguments separated by null bytes (\0) to avoid confusion with spaces inside arguments. tr swaps those null spaces to make it readable.

When you delete a file while a process holds it open, the data stays on disk. The file is gone from ls but the space is allocated until the process releases its file descriptor. In Linux, file is a link to a block of data, you are only removing the name. If the process has file descriptor open, data block will be kept alive. By redirecting to the FD in /proc, you're telling the kernel to wipe the actual data blocks while leaving the 'ghost' handle intact.
Find these with lsof | grep deleted. Fix without restarting the process:

> /proc/1234/fd/4

That truncates the held-open file to zero bytes through the process's own descriptor. Disk space reclaimed, process uninterrupted.

Routing: where packets actually go

The routing table is what your machine consults before sending any packet. It decides which network interface and which gateway to use.

ip route show

default via 192.168.1.1 dev eth0 proto dhcp src 192.168.1.100 metric 100
# local lane
192.168.1.0/24 dev eth0 proto kernel scope link src 192.168.1.100

The default line handles everything. It says "for any destination I don't have a specific rule for, send it to 192.168.1.1." That is your router. Every internet-bound packet passes through that rule.

Delete it and watch what happens:

ip route del default
ping google.com

connect: Network is unreachable

The machine is still on the network. It just cannot reach anything outside the local subnet because there is no default route. This is one of the first things to check when a server cannot reach the internet. The default route may simply not exist, perhaps because a VPN misconfigured it, or because someone ran ip route flush without knowing what it did.

ip route add default via 192.168.1.1   
# put it back

# Most useful debugging command for routing
ip route get 8.8.8.8

8.8.8.8 via 192.168.1.1 dev eth0 src 192.168.1.100

One line is the complete answer about which interface and gateway any packet would use to reach any destination.

/dev: the devices that are not devices

/dev/null, /dev/zero, /dev/random all live in the same directory as actual hardware. They have the same character device type marker. None of them are physical hardware.

/dev/null is a black hole. Write anything to it, and it is gone silently. Read from it, you get immediate end-of-file. This is what command > /dev/null 2>&1 does: stdout goes to the black hole, 2>&1 sends stderr to wherever stdout is going (also the black hole).
The numbers represent 'streams.' 1 is standard output (normal text), and 2 is standard error. 2>&1 means, take all the errors and send them into the same pipe where the normal text is going. Since that pipe leads to /dev/null, the command becomes perfectly silent.

# Silence all output but still check if it worked
some_noisy_command > /dev/null 2>&1
echo $?     # 0 means success, anything else means it failed

/dev/zero produces infinite zero bytes. Good for filling disks or creating blank test files:

fallocate -l 500M bigfile   # faster for large files
# or the /dev/zero version:
dd if=/dev/zero of=bigfile bs=1M count=500

df -h    # watch disk fill up
rm bigfile

fallocate is instant because it just tells the filesystem to reserve the space without actually writing anything to the disk. dd if=/dev/zero forces the CPU to actually churn out millions of zeros and write them one by one.

/dev/urandom produces cryptographically random bytes collected from hardware entropy:

cat /dev/urandom | tr -dc 'a-zA-Z0-9' | head -c 24
echo
# or the cleaner version:
openssl rand -base64 18

The thing that made this concrete for me: the same read() system call your code uses to open a text file reads random bytes from /dev/urandom. Same interface but completely different behavior underneath. That is the "everything is a file" idea became real.

/boot: where the kernel lives

ls /boot

vmlinuz-5.15.0-91-generic
initrd.img-5.15.0-91-generic
grub/
config-5.15.0-91-generic

vmlinuz is the compressed Linux kernel (vm = virtual machine, z- compressed like .zip). The actual operating system, stored as a file. initrd.img is a temporary RAM filesystem that acts as a bridge, which loads before the real filesystem mounts at boot. grub/ has the bootloader that lets you choose between operating systems.

cat /boot/config-$(uname -r) | grep CONFIG_EXT4

CONFIG_EXT4_FS=y

That file is the kernel build configuration. Every driver, every feature, listed. Most of it is cryptic without context but searching it teaches you what the kernel actually supports.

The only advice here: do not delete anything from /boot. Deleting vmlinuz means the computer cannot boot at all. Recovery requires a live USB. This is the one folder in the filesystem where curiosity should not turn into experiments.

Consider /etc is the brain and /proc is the nervous system, /boot is the heart of Linux.

/var/log: the system's diary

/var/log is where the machine writes what happened. Everything.

ls /var/log

auth.log    syslog    kern.log    dpkg.log
nginx/      apt/      journal/

auth.log records every login, failed or successful. syslog is the general system log. kern.log has kernel messages. dpkg.log records every package installed or removed and when.

Some logs are plain text files you can cat like syslog, while others live in the journal/ directory stores logs in a binary format to make them searchable and faster to filter. The journalctl command is used to read these binary logs.

tail -f /var/log/syslog           # live system events
journalctl -u nginx -f            # live logs for a specific service
journalctl -xe                    # recent journal with explanations
journalctl --since "1 hour ago"   # last hour of everything

When something breaks, look for the log file for the specific thing that broke. It almost always contains the exact error. If logs kept growing forever, they’d fill your entire disk. Linux uses a tool called logrotate that periodically takes the old log, compresses it in the .gz files, and eventually deletes it

/etc/systemd/system/ writing a service that survives crashes

Systemd starts everything on boot and manages services. Unit files in /etc/systemd/system/ tell it what to run.

I wanted a script running on boot that restarted if it crashed. Cron can run on boot with @reboot but has no crash recovery. A service file does:

nano /etc/systemd/system/myapp.service

[Unit]
Description=My app
After=network.target

[Service]
Type=simple
User=saumya
WorkingDirectory=/home/saumya/app
ExecStart=/usr/bin/node server.js
Restart=on-failure
RestartSec=5
StandardOutput=journal
StandardError=journal

[Install]
WantedBy=multi-user.target

After=network.target line is a safety check that tells Linux to not start this app until the network is actually up.
By default, system services want to run as root and can be a security risk. If the app gets hacked, User= makes sure the attacker is trapped in the user account and doesn't get access to keys. WorkingDirectory ensures that if the code looks for a file in ./config, it's looking in the right folder.

systemctl daemon-reload
systemctl enable myapp
systemctl start myapp
systemctl status myapp

Restart=on-failure is the important line. The process crashes, it comes back in 5 seconds because of RestartSec=5 which can be adjusted. You stop it with systemctl stop. The distinction between a crash and an intentional stop is handled automatically.

Logs for this service:

journalctl -u myapp -f

PATH why commands vanish when you break one variable

PATH tells the shell where to look for executables. When you type ls, the shell searches every directory in PATH until it finds a file named ls. If PATH is wrong, every command fails.

export PATH=""
ls

bash: ls: command not found

Everything is gone. The binaries still exist on disk but the shell just lost its map. It can be fixed with:

export PATH=/usr/bin:/bin:/usr/sbin:/sbin
ls    # back

Why this matters beyond the experiment: cron jobs run with a stripped PATH by default. If your script calls something in /usr/local/bin and cron's PATH does not include that directory, the cron job silently fails with "command not found" while the exact same script works fine in your terminal. Set PATH explicitly in your crontab:

PATH=/usr/local/bin:/usr/bin:/bin
* * * * * /home/saumya/myscript.sh

If you’re ever unsure where a command is actually coming from, use which. Running which ls shows the exact path the shell found. It also helps to verify if the map is working.

Permissions

Permissions in theory is chmod, rwx, octal numbers. In practice I kept getting them wrong until I broke something on purpose.

echo '#!/bin/bash
echo "this ran"' > myscript.sh
./myscript.sh

bash: ./myscript.sh: Permission denied

The file exists and the content is correct, but newly created files have no execute permission. Check with:

ls -la myscript.sh
# -rw-r--r-- 1 saumya saumya 30 myscript.sh

No x anywhere. Add it:

chmod +x myscript.sh
./myscript.sh
# this ran

Now the more interesting break. Remove execute from ls itself:

chmod -x /bin/ls
ls

bash: /bin/ls: Permission denied

ls still exists on disk. It just cannot be executed. From the shell's perspective it might as well not be there. Fix:

chmod +x /bin/ls

The three positions: owner, group, others mean the same permissions behave differently depending on who runs the command. When your web server returns 403 errors and logs say "permission denied," this is why. The files exist, the server is running, but www-data does not have the permission to read them.

chown -R www-data:www-data /var/www/html
chmod -R 755 /var/www/html

For a file: +x means you can run it; But for a directory: +x means you can view it using cd.
Linux works on Principle of Least Privilege(PoLP), hence avoid the temptation to run chmod 777.

Environment variables

A friend set a variable in ~/.bashrc. It worked in his terminal. His systemd service couldn't see it which is followed by hours of debugging.

Every process inherits environment variables from its parent. The terminal is started by the login process, which sources ~/.bashrc, which sets your variables. Systemd (PID 1) starts services. Systemd never read your ~/.bashrc. The variables your terminal has, the service has never heard of.

Your terminal reads ~/.bashrc and /etc/profile and has everything you configured. A cron job reads almost nothing. PATH is /usr/bin:/bin. Your tools in /usr/local/bin are invisible. A systemd service reads only what you explicitly declare in its unit file under [Service].

For services, declare variables in the unit file:

[Service]
Environment=NODE_ENV=production
EnvironmentFile=/etc/myapp/.env

For cron, set PATH at the top of the crontab or use absolute paths everywhere. For your own sessions, ~/.bashrc works. The confusion only happens when you expect one environment to be visible in another. In Linux, explicit is better than implicit. If a service needs a variable, tell the service directly.

Terminal shortcuts worth memorizing now

Ctrl+R searches command history interactively. Start typing any part of a command you ran before. This is faster than retyping long paths.

cd - goes to the last directory you were in. Jump between two locations without retyping either path.

!! repeats the last command. The classic use is sudo !! when you forgot sudo.

Ctrl+A and Ctrl+E jump to the start and end of the current line. Ctrl+W deletes the last word backwards. These three reduce terminal frustration noticeably.

When a config file edit breaks something and you want to know exactly what changed: diff /etc/nginx/nginx.conf.backup /etc/nginx/nginx.conf. Keep backups before editing important files. cp nginx.conf nginx.conf.bak takes two seconds.

I found two resources that can be helpful in the learning curve of Linux:

• Linux Journey : This has organized learning path that breaks down the complexities of the Linux operating system into digestible stages.

• Bandit : It is wargame designed to teach the fundamentals of the Linux command line. Visually looks boring, but is really interesting to follow through.

References

• Fireship: Linux in 100 Seconds : start here if you haven't

• Linux man pages : every file and command has one

• tldr pages : install with apt install tldr, use it for every new command

• Arch Wiki : the best Linux documentation on the internet, works for all distros regardless of what you're running

• explainshell.com : paste any command and it breaks down every flag visually

• The filesystem docs : official kernel documentation

Here's what that looks like in practice:

// Storing a function in a variable 
const greet = function(name) {
  console.log("Hello, " + name);
};

// Passing that variable to another function
function run(fn) {
  fn("Aditya"); // calls whatever was passed in
}

run(greet); // prints: Hello, Aditya

greet is just a value here. We handed it to run, and run called it. That's the whole mechanic. A callback is just a function passed to another function so that other function can call it later.

The word "callback" sounds formal, but the idea is simple: you're giving someone your phone number and saying "call me back when you're done." The function you pass is that phone number.

Passing Functions as Arguments

You've probably used callbacks already without noticing. addEventListener, setTimeout, Array.forEach, all of these take a function as an argument. That function is the callback.

// setTimeout takes a callback: runs it after 2 seconds
setTimeout(function() {
  console.log("Two seconds later");
}, 2000);

// forEach takes a callback: runs it once per item
["Manas", "Saumya", "Priya"].forEach(function(name) {
  console.log(name);
});

// addEventListener takes a callback: runs it on click
button.addEventListener("click", function() {
  console.log("Clicked");
});

Notice the pattern. You pass a function, something else holds onto it, and calls it at the right moment. You are not in charge of when it runs. That's kind of the point.

Why Async Even Needs Callbacks

JavaScript runs one thing at a time. There is no threading, no parallel execution of your code. One call stack, one line at a time. So when you need to wait for something, a network request, a file read, a timer, you have a problem: if you wait, everything else stops.

Think about a restaurant. The chef doesn't stand at the kitchen pass staring at your food until you come collect it. They put it on the counter, ring a bell, and move on to the next order. You come when you're ready. The kitchen keeps running.

Callbacks are that bell. You hand the runtime a function and say: when the thing is done, run this. Meanwhile, the rest of your code keeps going.

// SYNCHRONOUS: second log waits for the whole loop
console.log("start");
for (let i = 0; i < 1000000000; i++) {} // browser hangs here
console.log("end");

// ASYNC WITH CALLBACK: second log runs immediately
console.log("start");
setTimeout(function() {
  console.log("I ran later"); // runs after 2s, non-blocking
}, 2000);
console.log("end"); // this prints before the callback

Running that second block, you'll see "start", then "end", then two seconds later "I ran later". The callback didn't block anything.

Common Places Callbacks Show Up

Callbacks are everywhere. Once you start seeing the pattern, you can't unsee it. Here are the places you'll hit them most as a beginner.

// array methods: forEach, filter, map
const scores = [42, 88, 61, 95, 37];

// forEach: run callback for each item
scores.forEach(function(score) {
  console.log(score);
});

// filter: keep items where callback returns true
const passing = scores.filter(function(score) {
  return score >= 60;
}); // [88, 61, 95]

// map: transform each item, return new array
const grades = scores.map(function(score) {
  return score >= 60 ? "Pass" : "Fail";
}); // ["Fail", "Pass", "Pass", "Pass", "Fail"]

// Event Listeners
const btn = document.getElementById("submit");

// callback runs every time the button is clicked
btn.addEventListener("click", function(event) {
  event.preventDefault();
  console.log("Form submitted");
});

// fetch API
// fetch uses .then()  which takes a callback
fetch("/api/users/1")
  .then(function(response) {
    return response.json(); // callback 1: parse response
  })
  .then(function(data) {
    console.log(data.name); // callback 2: use the data
  });

Arrow functions are common shorthand for callbacks: scores.filter(s => s >= 60) does the same thing as the function version above. Same idea, fewer keystrokes.

Writing Your Own Function That Takes a Callback

It helps to write one yourself, even if it's small. Once you've built a function that accepts and calls a callback, the whole model locks in.

// greetUser takes a name and a callback
function greetUser(name, callback) {
  console.log("Fetching user data...");
  callback(name); // call whatever was passed in
}

// Two different behaviours, same greetUser function
greetUser("Saumya", function(name) {
  console.log("Hello, " + name + "!"); // "Hello, Saumya!"
});

greetUser("Manas", function(name) {
  console.log("Welcome back, " + name); // "Welcome back, Manas"
});

greetUser doesn't care what the callback does. It just calls it with the name. The caller decides the behaviour. That flexibility is exactly why callbacks are so useful — you write the structure once, plug in different logic as needed.

A real-world version of this is Array.sort. The sort algorithm is built into JavaScript, but you pass a callback that defines how to compare two items. You bring the comparison logic; sort brings the algorithm.

// sort with custom comparator callback
const users = [
  { name: "Manas",  age: 28 },
  { name: "Saumya", age: 24 },
  { name: "Priya",  age: 31 },
];

// sort needs to know HOW to compare — you provide that
users.sort(function(a, b) {
  return a.age - b.age; // youngest first
});

The Nesting Problem

Here's where callbacks get genuinely painful. Suppose you need to do three async things in sequence: fetch a user, then fetch their orders, then fetch the details of the first order. Each step depends on the previous one. With callbacks, you end up putting them inside each other.

getUser(1, function(user) {

  getOrders(user.id, function(orders) {

    getOrderDetails(orders[0].id, function(details) {

      console.log(details); // finally, the thing we wanted

    });
  });
});

Three levels. Now add error handling at each step. Add a fourth request. The indentation becomes a visual pyramid, and the logic is buried inside it. This has a name: callback hell.

The nesting itself is not wrong, it works. The problem is readability. Errors are hard to catch at the right level. Adding a step means nesting deeper. Debugging means tracing through a maze of closing braces. It gets old fast.

Callback hell is why Promises and async/await exist. They solve the same coordination problem with much flatter code. That's the next part of this series.

What Callbacks Actually Give You

REFERENCES:

• MDN Web Docs : Callback function

• JavaScript.info : Introduction: callbJavaScript.info : Introduction: callbacks

• JavaScript Asynchronous Programming and Callbacks

• Callback Hell