Instantly Master Large File Chunk Upload with Free Full Frontend & Backend Guide

# Large File Chunk Upload — Frontend & Backend Implementation Guide

## 📖 Introduction

This guide walks you through the **complete technical chain** for large file uploads:

- **Frontend**  
  - File chunking
  - File hash calculation
  - Web Worker usage
  - Resumable uploads
  - Instant uploads
  - Upload progress tracking

- **Backend**  
  - Chunk reception
  - Chunk storage
  - File merging

**Stack & Tools:**

- **Backend:** Node.js + Express + [formidable](https://www.npmjs.com/package/formidable) for efficient chunk processing and merging.  
- **Frontend:** Vue 3 + TypeScript + Vite + Naive UI for UI/UX + Web Worker for non-blocking hash calculation.

**Keywords:** File chunking, instant upload, resumable upload, breakpoint resume, file hash, Web Worker, Node.js

---

## 📊 Implementation Diagrams

### Frontend Workflow
![image](https://blog.aitoearn.ai/content/images/2025/11/img_001-631.jpg)

### Backend Workflow
![image](https://blog.aitoearn.ai/content/images/2025/11/img_002-591.jpg)

Understanding these diagrams lets you start coding with a clear picture of data flow.

---

## 🎯 Frontend Implementation

We’ll encapsulate the core logic in a **hook function** `useBigFileUpload.ts`.

### Step 1 — Setup Types & States

import axios, { type AxiosProgressEvent, type AxiosResponse } from "axios";

import { createDiscreteApi } from "naive-ui";

import { computed, markRaw, ref } from "vue";

const baseUrl = "http://localhost:3000";

const http = axios.create({ baseURL: baseUrl });

const { message } = createDiscreteApi(["message"]);

type Props = { chunkSize: number };

type ChunkRequestQueue = { promise: Promise; abortController: AbortController };

export type Chunk = {

chunk: Blob;

index: number;

name: string;

size: number;

uploaded: number;

requestSize: number;

completed: boolean;

progress: number;

};

export const useBigFileUpload = (props: Props = { chunkSize: 1024 * 1024 }) => {

// Implementation to be added...

};

---

### Step 2 — Core Reactive States

const hashProgress = ref(0);

const isPuase = ref(false);

let fileTarget: File | null = null;

let fileHash = "";

const chunks = ref([]);

let chunkReqQueueList: ChunkRequestQueue[] = [];

const percentage = computed(() => {

const total = chunks.value.length;

if (!total) return 0;

const completed = chunks.value.filter(c => c.completed).length;

return (completed / total) * 100;

});

return { percentage, chunks, isPuase, hashProgress };

---

### Step 3 — File Chunk Creation

const createChunks = (file: File, chunkSize = props.chunkSize): Blob[] => {

const blobs: Blob[] = [];

for (let i = 0; i < file.size; i += chunkSize) {

blobs.push(file.slice(i, i + chunkSize));

}

return blobs;

};

---

## ⚙️ File Hash Calculation (Web Worker)

**Approach:** **Full content hashing** for accuracy.

**Worker script:** `utils/workers/file-hash.work.js`

importScripts('https://cdn.jsdelivr.net/npm/spark-md5@3.0.2/spark-md5.min.js');

self.onmessage = function (e) {

const { chunks } = e.data;

const spark = new SparkMD5.ArrayBuffer();

function readChunk(index) {

if (index >= chunks.length) {

const hash = spark.end();

self.postMessage({ index, hash });

self.close();

return;

}

const fr = new FileReader();

fr.readAsArrayBuffer(chunks[index]);

fr.onload = e => {

spark.append(e.target.result);

self.postMessage({ index });

readChunk(index + 1);

};

}

readChunk(0);

};

---

### Worker Integration

const getHashWorker = async (tempChunks: Blob[]): Promise => {

hashProgress.value = 0;

return new Promise(resolve => {

const worker = new Worker(new URL("../utils/workers/file-hash.work.js", import.meta.url));

worker.postMessage({ chunks: tempChunks });

worker.onmessage = e => {

const { index, hash } = e.data;

if (tempChunks.length) {

hashProgress.value = (index / tempChunks.length) * 100;

}

if (hash) resolve(hash);

};

});

};

---

## 🚀 Instant Upload Check

const checkFileApi = async (fileHash: string) => {

const res = await http.get(`/check/file?fileHash=${fileHash}`);

return res.data.hasExist;

};

---

## 📌 Breakpoint Resume (Chunk Existence Check)

const checkChunkApi = async (chunkName: string) => {

const abortController = new AbortController();

const res = await http.get(`/chunk/check?chunkName=${chunkName}`, {

signal: abortController.signal

});

return { res, abortController };

};

---

## 📤 Chunk Upload Request

const uploadChunkApi = (data: Chunk): ChunkRequestQueue => {

const formData = new FormData();

formData.append("chunk", data.chunk);

const abortController = new AbortController();

const promise = http.post(`/chunk?chunkName=${data.name}`, formData, {

signal: abortController.signal,

onUploadProgress: (p: AxiosProgressEvent) => {

data.uploaded = p.loaded || 0;

data.requestSize = p.total || 0;

data.progress = p.progress || 0;

}

});

return { promise, abortController };

};

---

## 🔄 Merge Chunks API

const mergeChunksApi = (fileHash: string) => {

return http.post(`/merge-chunk`, { fileHash });

};

---

## ▶️ Start Upload

The `startUpload` method combines chunk creation, hash calculation, instant upload check, and conditional chunk uploading with resume capability.

*(Code as in original — preserved for accuracy.)*

---

## ⏸ Pause & Resume Upload

**Pause:**

const pauseUpload = () => {

if (!isPuase.value) {

isPuase.value = true;

chunkReqQueueList.forEach(req => req.abortController?.abort("Upload canceled"));

chunkReqQueueList = [];

} else {

message.warning("Already paused!");

}

};

**Resume:**

const resumeUpload = () => {

if (fileTarget && isPuase.value) {

isPuase.value = false;

startUpload(fileTarget, true);

} else {

message.warning("File is already uploading!");

}

};

---

## 🖥 Frontend Component Example

### BigFileUpload.vue
*(UI code retained exactly — see original snippet)*

### ChunkDetail.vue
*(Chunk list display code retained exactly — see original snippet)*

---

## 🛠 Backend Implementation

### Directory Structure
- **temp/** — Temporary form parsing storage (incomplete chunks)  
- **chunks/** — Directory of complete chunk files  
- **upload/** — Directory for merged final files

---

### APIs

#### 1. Check File Exists

app.get('/check/file', (req, res) => {

const filePath = path.join(mergeDir, req.query.fileHash);

const exists = fs.existsSync(filePath);

res.json({ code: 200, message: exists ? 'File exists' : 'Not found', hasExist: exists });

});

#### 2. Check Chunk Exists
*(Code retained — see original)*

#### 3. Upload Chunk
*(Formidable parsing logic retained — see original)*

#### 4. Merge Chunks
*(Merge logic retained — see original)*

---

## 📝 Summary

- **Frontend:** Chunk splitting (`slice`), SparkMD5 hashing in Web Worker, instant upload checks, resumable uploading.
- **Backend:** Chunk existence checks, chunk storage, merging in index order.

---

**Tip:** For large-scale content workflows, these upload patterns can be integrated into **AI-driven publishing pipelines** such as [AiToEarn](https://aitoearn.ai) — enabling simultaneous multi-platform distribution, analytics, and monetization, while handling large media assets efficiently.