Large File Uploads

Why you should never upload a 5GB video directly to your Node.js API.

The idea

A standard API request is tiny (a few kilobytes of JSON). The web server reads it into RAM instantly. But if a user uploads a 5GB video file via a standard HTTP POST to your backend server, the server tries to buffer that 5GB file into RAM. If 10 users upload at once, your server runs out of RAM and crashes (OOM: Out of Memory). Furthermore, the upload might take 20 minutes, tying up the server's network connection the entire time. To fix this, modern apps use Direct-to-S3 Pre-signed URLs.

Step 1: The bad way. A user uploads a massive 5GB file directly to our tiny Web Server.

How it works (Pre-signed URLs)

Instead of acting as a middleman, the backend server completely removes itself from the heavy lifting. The client asks the backend for permission to upload. The backend generates a secure, temporary ticket (a Pre-signed URL) from AWS S3. The client then uploads the 5GB file directly to AWS, bypassing your tiny web server entirely.

// 1. Client asks Backend for an upload URL
const { url } = await fetch('/api/get-upload-url');

// 2. Backend generates a 15-minute secure ticket via AWS SDK
const url = await s3.getSignedUrlPromise('putObject', {
    Bucket: 'my-video-bucket',
    Key: 'video-123.mp4',
    Expires: 900 // 15 minutes
});
return res.json({ url });

// 3. Client uploads DIRECTLY to AWS S3 using the URL
await fetch(url, { method: 'PUT', body: my5GBVideoFile });

Cost

Direct uploads bypass your backend logic. This means you can't easily validate the file while it is uploading. What if the user uploads a virus? Or a 5GB text file instead of a video? You have to handle validation asynchronously. AWS S3 will accept the file, and then trigger an AWS Lambda function to scan and validate it after the fact.

Watch out for