Cold Storage Retrieval Timeouts

When asynchronous realities break synchronous expectations.

The idea

If a user clicks "Download 2018 Tax Return", and that file is in AWS S3 Glacier (Deep Archive), it will take 12 hours for the file to be readable. If your web app tries to fetch the file synchronously, the HTTP request will simply hang and eventually Time Out, crashing the web thread and frustrating the user.

Step 1: User clicks download on an archived file.

How it works (Async Retrieval & Webhooks)

You cannot treat Cold Storage like a normal database. Retrieving data from it requires an asynchronous architecture. You issue a "Restore Request", immediately tell the user "We are working on it", and then wait for the storage provider to send you a webhook when the file is finally ready.

# Correct Asynchronous Flow for Cold Storage

@app.route('/download/tax-2018')
def download_old_tax():
    file_metadata = db.get("tax-2018")
    
    if file_metadata.storage_tier == "GLACIER":
        # 1. Trigger the restore job asynchronously
        s3.initiate_restore(file_metadata.s3_path, duration_days=1)
        
        # 2. Return immediately (NO TIMEOUT!)
        return render_template("pending.html", 
            message="We are retrieving this from the archive. We will email you in 12 hours.")
            
    # If it was in Hot storage, serve immediately
    return send_file(file_metadata.s3_path)

# 3. 12 Hours later, AWS calls our Webhook
@app.route('/webhook/s3_restore_complete')
def s3_webhook():
    file_path = request.json['path']
    user_email = db.get_owner_of(file_path)
    send_email(user_email, "Your file is ready to download!")

Cost

This adds significant UX complexity. You need UI states for "Retrieving...", webhooks, and email/push notifications. However, it completely prevents server thread exhaustion (which would happen if threads blocked waiting for 12 hours).

Watch out for