Understanding File Descriptors (FD) in Linux

At a previous job, one of my colleagues — a senior engineer of the same age as me — once insisted that we needed to scale out our servers because the system had supposedly hit its FD limit. It turned out he misunderstood FDs at the user level rather than the process level. That incident made me realize this confusion is fairly common, so I took the time to educate the team. Remembering that experience, I’m sharing this write-up as part of my notes on file descriptors.

In Linux operations, it’s not unusual to hear that “too many files are open” or “the FD limit must be raised.” But unless you understand exactly what a file descriptor is, you might end up adding unnecessary servers — or worse, overlook a real sign of trouble. This post walks through the basics of FDs, how to check them, and what to watch out for.

1. What is a File Descriptor?

In Linux, everything is treated as a file. Actual files, directories, sockets, pipes, and devices are all managed using file descriptors (FDs) — simple integer values.

If the system runs out of FDs, no additional file or socket can be opened, which eventually leads to service outages.

2. Checking System-Wide FD Usage

System-wide FD usage is exposed via /proc/sys/fs/file-nr:

cat /proc/sys/fs/file-nr
37248   0   13123092

• First value → number of allocated FDs (in use)
• Second value → number of free FDs
• Third value → maximum number of FDs allowed system-wide

Kernel version differences

• Linux 2.6 ~ 4.x: the second value is always 0. Free FDs were not tracked at the kernel level.
• Linux 5.x and later: the second value reflects the actual number of free FDs, thanks to changes in FD cache management.

On Ubuntu, kernel versions vary by release and HWE stack, so always confirm the kernel version:

uname -r

Examples:

• 4.15.0-210-generic → kernel 4.x, free value always 0
• 5.15.0-122-generic → kernel 5.x, free value displayed correctly

3. Per-Process FD Limits

Each process has its own FD limit. You can check it with ulimit:

ulimit -Sn   # soft limit
ulimit -Hn   # hard limit

• Soft limit → the active threshold enforced by the shell or service
• Hard limit → the maximum ceiling; cannot be raised beyond this without explicit configuration

Example:

ulimit -Sn
30480

→ Each process can open up to 30,480 FDs simultaneously.

4. Measuring Actual Usage

To see actual FD usage, use lsof.

By user

lsof -u <username> | wc -l
50867

→ All processes under this user combined are holding about 50,000 FDs.

By process

lsof -p <PID> | wc -l
300

→ Process <PID> is using ~300 FDs. Well below the per-process limit (e.g. 30,480).

5. When Do Problems Happen?

FD-related outages usually occur in two scenarios:

1. System-wide exhaustion
   The first value in /proc/sys/fs/file-nr (in-use FDs) reaches the third value (max).
   No new file or socket can be opened, causing service failure.
2. Per-process exhaustion
   A process hits its ulimit -Sn value.
   Errors like Too many open files appear — common in Java applications with runaway DB connections or socket handles.

6. Common Misunderstandings

• “FD limits apply per user” → Incorrect. Limits are enforced per process. A single user can run multiple processes, each with its own FD limit.
• “If lsof -u is higher than ulimit, there’s a problem” → Misleading. lsof -u shows totals across all processes for that user; ulimit applies individually to each process.

7. Summary

• FDs are the basic unit for all resources in Linux.
• /proc/sys/fs/file-nr → system-wide FD status; free field depends on kernel version.
• ulimit -Sn → per-process FD limit.
• lsof -p <PID> → FD usage of a specific process.
• Most issues come from hitting per-process or system-wide limits.
• When FDs run out, socket connections fail and services stop.