No space left on device, but df shows free space (out of inodes)

What this is#

The confusing cousin of a full disk: writes fail with "No space left on device", your site or database misbehaves exactly as if the disk were full, but df -h shows gigabytes free. What actually ran out is inodes, and because almost nobody checks for it, it's one of the most-missed causes of a broken server.

Confirm it in five seconds:

df -i

If IUse% is at (or near) 100% on the affected filesystem, this page is your problem.

What an inode is#

Every file and directory on a filesystem is tracked by an inode, the record holding its metadata and where its data lives. A filesystem is created with a fixed budget of them (roughly one per 16 KB of disk on default ext4). Normal use never gets close, but millions of tiny files spend inodes while barely using any bytes, which is how you end up "full" with free space: the byte tank has room, the file-count tank is empty.

Find the hoard#

You're looking for the directory with a ridiculous number of files, not big ones. GNU du counts inodes directly, drill down largest-first exactly like a disk hunt:

du --inodes -xd1 / | sort -n

Re-run it on the worst directory (du --inodes -xd1 /var | sort -n, and so on) until you hit the pile. The usual suspects:

• PHP session files in /var/lib/php/sessions, the classic, when session garbage collection is off or the site is busy or being spammed.
• Cache directories: app caches, WordPress cache plugins, composer/npm caches, thumbnail caches.
• Mail queues and mailboxes: a cron job erroring every minute for a year quietly writes half a million messages to /var/mail/root; a stuck postfix queue does the same in /var/spool.
• Temp and job files from an app or queue worker that creates but never deletes.
• Docker layers and volumes holding many-small-file images.

Clean it up, the right way#

A directory with millions of entries breaks the naive tools: rm * fails with "Argument list too long", and even ls can hang. Use find, which streams:

find /var/lib/php/sessions -xdev -type f -delete

(Point it at the guilty directory, and only at files you've confirmed are disposable, sessions, caches, and queue debris usually are; when unsure, ask first, same rule as the what-not-to-delete list.) For very large piles this can take a while, that's normal, let it run.

Stop the producer#

Deleting the pile without fixing the source buys you weeks, not a cure:

• PHP sessions: make sure session garbage collection runs (session.gc_probability/gc_divisor, or the distro's session-clean cron/timer).
• Caches: give them a TTL or a max size; most frameworks and cache plugins have one that's just not enabled.
• Cron mail: fix the erroring job, and set MAILTO="" on jobs whose output nobody reads.
• Queues/temp files: make the app clean up after itself, or add a tmpfiles/cron cleanup for its directory.

Worth knowing#

• You can't add inodes to an existing filesystem by tuning, the budget is set when it's created. But growing the disk grows the budget: when you upgrade the plan or add storage and the filesystem expands, the inode count expands proportionally with it. It's still the wrong fix for a runaway producer, right fix for a genuinely file-heavy workload (huge maildirs, node_modules farms).
• Add df -i to your mental checklist right next to df -h, the pair takes ten seconds and covers both ways a disk can be "full".

Still need help?#

You can open a support ticket. So we can help on the first reply, it's worth mentioning:

• the VPS hostname or IP,
• the df -i output and the directory the inode hunt pointed at,
• what appears to be producing the files.

Related questions#

• "Why does it say no space left on device when df shows free space?"
• "What is an inode and why did I run out?"
• "How do I find which directory has millions of files?"
• "How do I delete millions of files (argument list too long)?"
• "Does upgrading my disk add more inodes?"