Duplicati fails after large file: blocksize, timeouts, and quota explained
By Daniel Brooks Updated 8 min read
On this page (12 sections)
- The short version
- Why Duplicati backup keeps failing after a large file is added
- Where to find the Duplicati log that names the failure after a large file
- How the default blocksize causes large-file failures and what to do
- Dblock size and its trade-offs
- How to check and adjust timeout settings for large uploads
- Calculating local temp space needs for backup
- How to check backend quota and local disk space when backups fail
- How Repair works and when to use it
- What to do if failures occur during the compact phase
- When to exclude, split, or throttle large files
- Questions people still ask
In short: Duplicati backup failures after adding a large file mainly happen because the default blocksize creates a huge database, or timeouts slow uploads, or storage limits are reached. Check your Duplicati error log to find the exact cause. Then adjust your settings or storage accordingly.
Part of our guide on versioning and data retention
This guide shows how to read Duplicati's errors to tell if large-file backup fails from blocksize, timeout, or storage limits, then fix each.
| Default blocksize | 100 KB |
|---|---|
| Backend max object size | Varies, often 4-5 GB or less |
| Default HTTP timeout | 300 seconds (may be overridden by backend SDK) |
| Local temp space needed | Approximately blocksize × (concurrent uploads + async queue) |
The short version
- Read Duplicati's log to identify the failure cause when large files are added
- Raise blocksize for backups that include multi-gigabyte files to avoid database bloat
- Increase timeout settings mindful of backend limits and upload speed
- Confirm backend quota and local temp disk space are sufficient
- Use Repair to fix database mismatches, not to resume interrupted backups
Why Duplicati backup keeps failing after a large file is added
Adding a large file to your Duplicati backup can cause the job to hang, time out, or fail with errors. This usually happens because the default blocksize is very small (100 KB) which creates millions of blocks for a multi-gigabyte file.
Each block generates a database row. When the block count explodes, the local SQLite database becomes huge and slow. This can cause the backup to stall or run out of temp space.
Timeouts also cause failures. Uploading many small blocks can extend the total time, leading to HTTP operation timeouts or backend SDK timeouts that Duplicati's timeout setting does not control.
Storage limits are another common cause. Your cloud backend might reject uploads exceeding its per-object size limit, or your local disk might lack enough free space for temporary chunk assembly. We cover how to compact virtualbox's vdi file size in its own article.
You must check the log to see which cause applies, as the symptoms overlap but the fixes differ.
- Default blocksize too small: millions of blocks create a huge local database
- Timeouts from long upload times or backend SDK limits
- Backend per-object size limits prevent storing large chunks
- Local temp disk space insufficient for chunk assembly
Where to find the Duplicati log that names the failure after a large file
Open Duplicati's web interface and go to the backup job's log section.
Select 'Show log' then 'Live' or 'Retry' to view detailed messages from the current or last run. It helps to understand delete appdata local temp before going further.
Look for error messages mentioning 'block', 'timeout', 'RequestEntityTooLarge', or disk write failures.
Timeout errors typically say 'Operation timed out' or 'HTTP request timeout'.
Disk space errors mention 'System.IO.IOException' or 'write failure' indicating local storage problems. For the detail, see our notes on get deleted files back.
- Open Duplicati web interface
- Click on your backup job and select 'Show log'
- Choose 'Live' or 'Retry' log view
- Search for errors mentioning block, timeout, quota, or disk space
- Note exact error messages and timestamps for troubleshooting
How the default blocksize causes large-file failures and what to do
Duplicati splits files into blocks of 100 KB by default. A multi-gigabyte file breaks into millions of blocks.
Each block adds a row to the local SQLite database, which can grow very large and slow to query, making the backup hang or fail.
Raising the blocksize reduces the number of blocks and database rows, improving performance for large files. We go through keeping files intact during migration step by step elsewhere on the site.
Changing the blocksize requires starting a new backup set because the blocksize is fixed per backup.
Set blocksize based on your largest files and backend limits. Many backends limit uploads to 4-5 GB per object. Choose a blocksize under that limit to avoid rejections.
Dblock size and its trade-offs
Duplicati groups blocks into dblocks for upload and storage. The default dblock size is about 50 MB. Before you commit to anything, it is worth looking at difference between sync and backup iphone photos.
Larger dblocks mean fewer requests but longer uploads per chunk. Smaller dblocks reduce retry cost if an upload fails but increase the number of requests.
Backend storage providers enforce maximum object sizes. Choose a dblock size below that ceiling.
Adjusting dblock size affects how much data is retried on failure but does not fix the database bloat caused by small blocksize.
Blocksize and dblock size are separate settings with different effects.
How to check and adjust timeout settings for large uploads
Duplicati's '–http-operation-timeout' defaults to 300 seconds. Many backends implement their own SDK timeouts that this setting does not override.
Long uploads of large dblocks or many blocks can hit these timeouts, causing failures.
Increase the timeout if uploads take longer than the default. Start by doubling to 600 seconds and adjust based on your upload speed and file size.
Watch the log for 'Operation timed out' or HTTP timeout errors to confirm.
If backend SDK timeouts cause failures, consider splitting files or reducing dblock size to shorten individual upload times.
Calculating local temp space needs for backup
Duplicati assembles dblocks in a temp folder before upload, requiring enough free disk space.
The peak temp space needed roughly equals blocksize × (concurrent upload limit + async upload queue length) plus space for restore and compact operations.
For example, with a 50 MB dblock size and 4 concurrent uploads plus a queue of 2, peak temp space is about 50 MB × 6 = 300 MB, plus extra for restore and compact.
If temp space is insufficient, backups fail with disk write errors.
Use OS tools to check free space on the temp folder drive and free space accordingly.
How to check backend quota and local disk space when backups fail
Cloud storage providers limit your quota and may reject uploads when full.
Log into your provider's dashboard to check used and available storage.
Verify your local drive where Duplicati stores temp files has enough free space based on the calculation above.
Cleaning temp folders or expanding quota/disk space can fix many failures without changing Duplicati settings.
If you see errors about quota exceeded or disk full in the log, start by freeing space.
- Check cloud provider storage dashboard for quota usage
- Check free space on local temp folder drive
- Clear temp folder if space is low
- Increase cloud quota or local disk space if needed
- Retry backup after freeing space
How Repair works and when to use it
Repair synchronizes the local Duplicati database with the remote file list from your backend.
It fixes mismatches caused by interrupted backups or manual file changes.
Repair does not resume interrupted backups; interrupted jobs resume automatically on the next run.
Running Repair can take a long time on large databases, sometimes hours, depending on database and network speed.
Use Repair only when logs show missing or extra remote files, not just because a backup failed.
- Open Duplicati web interface
- Select the affected backup job
- Click 'Repair' in the job menu
- Wait for repair to complete
- Run backup again to resume
What to do if failures occur during the compact phase
Failures can happen during database compaction after large file uploads.
Compaction requires significant disk space and time to rewrite the database.
Ensure enough local disk space and patience while compaction completes.
If compaction fails repeatedly, consider backing up with a fresh database or adjusting blocksize to reduce database size.
Check logs for compaction error details to diagnose.
When to exclude, split, or throttle large files
Sometimes large files cause repeated failures despite tuning.
Exclude large files from your backup set using filters and back them up separately.
Manual splitting before backup can help fit files into a manageable blocksize and meet backend limits.
Duplicati splits files into blocks internally, so manual splitting mainly reduces block count but fragments deduplication.
Throttling upload speed can reduce timeout errors on slow or unstable connections but increases backup time.
- Exclude large files to avoid repeated failures
- Split files manually before backup to reduce block count
- Throttle upload speed to reduce timeout risk
Questions people still ask
How can I tell if failure is due to blocksize or timeout?
Check your log for errors mentioning blocks or operation timeout. Blocksize failures often say 'database busy' or show huge database size. Timeout errors mention 'Operation timed out' or HTTP timeout exceptions.
Can I change blocksize mid-backup without losing data?
No. Blocksize is fixed per backup set. To change it, start a new backup with the desired blocksize. Mixing blocksize settings causes errors.
What timeout value works for slow uploads?
Try doubling the default 300 seconds to 600 seconds, then adjust based on your upload speed and file size. Watch logs to avoid premature timeouts.
How much local disk space does Duplicati need for temp files?
Duplicati needs temp space about blocksize × (concurrent uploads + async queue length) plus extra for restore and compact operations. Check your settings to calculate.
Is excluding large files bad for backup completeness?
Excluded files aren't backed up in that job. Backup them separately to avoid losing data.
Does throttling upload help with failures?
Yes, slowing uploads reduces timeout risk on slow or unstable networks but increases total backup time.