Coder Legion - Recent posts tagged linux-commands

Linux Fundamentals for DevOps: Filesystem

Mon, 27 Apr 2026 16:45:41 +0000

When you SSH into a server to fix a broken application or install a new tool, the absolute worst thing you can do is guess where files belong. Linux does not put files in random places; it follows a strict, predictable map.

Understanding this map means you know exactly where to look when an application crashes, when a disk fills up, or when you need to configure a container.

If you run ls / on your server, you'll see a massive wall of folders.

As a DevOps engineer, you don't need to memorize every single one. Here are the crucial directories you will actually use, what they do, and the proof of why you should care about them.

`/etc` — The Control Room (Configurations)

You need to change the port your web server listens on, block a user from SSHing into the server, or configure a systemd service.

Historically, /etc stood for "et cetera," but today it is best understood as Editable Text Configurations. Every server, daemon, and background service stores its settings here.

Real scenario: Change the port Nginx listens on
Your app is running on port 80 but you need it on 8080 for a staging environment.

# Step 1: Find the config
cat /etc/nginx/nginx.conf

# Step 2: Edit it
nano /etc/nginx/nginx.conf
# Change: listen 80  →  listen 8080

# Step 3: Reload (no downtime!)
systemctl reload nginx

`/var` — The Growing Data Hub

This directory stands for "variable." It is designed to hold data that actively grows and changes while the system is running. If your server suddenly triggers a "Disk Space 100%" alert, /var is almost always the culprit.

We split this into two critical sub-directories:

/var/log (System and Application Logs): Your web application returned a 500 Error, or a database failed to start. This is the very first place you check.

# Watch errors as they happen, live
tail -f /var/log/nginx/error.log

# Search last 100 lines for "error" keyword
tail -n 100 /var/log/syslog | grep -i error

# See how much log data has accumulated
du -sh /var/log/

/var/lib (State Information and Databases): This is where applications save their actual heavy data. It is where PostgreSQL stores its database files, and where Docker stores its images and containers.

# "Why is my disk 80% full after installing Docker?"
du -sh /var/lib/docker

# Prune unused Docker images to reclaim space
docker system prune -a

# Where PostgreSQL stores its actual database files
ls /var/lib/postgresql/

NOTE: On modern Linux (Ubuntu 20.04+, Debian 10+), /lib is not a real directory anymore. You can see /lib and /usr/lib has common files

`/home` and `/root` — User Territories

You need to grant a new developer SSH access to a server, or you need to authenticate your server to pull code from a private Git repository.

These directories hold user-specific files. /home contains folders for standard users (e.g., /home/saeed), while /root is the isolated home directory for the superuser.

The DevOps importance: configuration files starting with a dot that control how tools behave per-user. Your .bashrc (shell settings), .kube/config (Kubernetes cluster access credentials), and .ssh/ (SSH keys) are all stored here.

# Grant a new developer SSH access
# They send you their public key. You do this:
cat dev-alice-key.pub >> ~/.ssh/authorized_keys

# Verify it was added
tail -n 3 ~/.ssh/authorized_keys

# Alice can now SSH in — no password needed

`/proc` — The Matrix (A Window Into the Kernel)

You are running a Docker container and need to know exactly how much memory the kernel is allowing it to use, or you need to see the exact environment variables a running process received. Every running process gets a numbered folder here (its Process ID, or PID) containing its live runtime data.

/proc The files aren't stored on disk — they don't exist until you read them. It is a "virtual filesystem" created in memory by the Linux kernel. . The moment you open /proc/1234/status, the kernel generates the content on the fly. It's a live feed into what's actually happening right now.

# Find the PID of your container's main process
docker inspect --format='{{.State.Pid}}' my-container

# See every env variable it received at startup
cat /proc/1234/environ | tr '\0' '\n'

# See what files/sockets it has open right now
ls -la /proc/1234/fd

# See exact memory usage the kernel is tracking
cat /proc/1234/status | grep VmRSS

# How long has this server been running?
cat /proc/uptime

# CPU info — useful in CI/CD pipelines
cat /proc/cpuinfo | grep "model name" | head -1

# Current memory usage
cat /proc/meminfo

`/dev` — Everything is a File

Linux's philosophy is: "if it can be read from or written to, it's a file." Your hard drive? A file. A USB drive? A file. Even nothingness and randomness are files. /dev is where all these hardware-and-virtual devices live as file objects. Your hard drives show up here (like /dev/sda or /dev/nvme0n1). But it also holds special virtual devices that are incredibly useful for scripting.

You are writing a cron job and want to completely hide its output so it doesn't spam your logs, or you are attaching a new block storage volume to your cloud instance.

#/dev/null — The Black Hole                                                                                                                                                                                                                                                                                                                                                        
# This cron job spams your mail. Make it silent:
# The "2>&1" part also redirects error output to null

./noisy-backup-script.sh > /dev/null 2>&1

# Useful breakdown:
#   >           redirects stdout
#   /dev/null   the black hole
#   2>&1        also send stderr to wherever stdout goes

`/usr` — The Real Home of Installed Software

You downloaded a pre-compiled binary (like terraform or helm) and want to install it so that anyone on the server can use it.

/usr (Unix System Resources) is where the vast majority of your installed applications, libraries, and tools actually live.

The DevOps importance: When you manually download and install a binary, you should place it in /usr/local/bin. This specific folder is meant for tools managed by the system administrator, ensuring they don't get overwritten when the system package manager (like apt or yum) runs its automated updates.

# Move a downloaded binary to the standard location for user-installed tools
mv terraform /usr/local/bin/terraform

`/opt` — Third-Party Add-ons

You are installing a heavy monitoring agent like Datadog, or a proprietary vendor application.

Linux reserves /opt (Optional software) for self-contained, third-party applications. Instead of scattering files across /usr and /etc, these monolithic tools keep everything neatly sandboxed inside their own dedicated folder here.

`/tmp` — The Throwaway Folder

You need to download a zip file, extract a shell script, run it, and never think about it again.

The /tmp directory is for temporary files. It is completely wiped clean every time the server reboots, so you should never put persistent or important data here. It also usually has open write permissions, making it the perfect staging ground for quick tasks.

Blog Summary

This guide breaks down the essential directories every DevOps engineer needs to know—like /etc for configurations, /var for logs, and /proc for live system processes. Complete with real-world use cases and simulated terminal outputs, this post serves as a practical map for quickly troubleshooting crashes, managing servers, and finding exactly what you need without getting lost.

Linux Fundamentals For DevOps: Linux core mechanism

Mon, 27 Apr 2026 16:45:13 +0000

Knowing where files live is only half the battle. To be an effective DevOps engineer, you need to know how Linux actually executes programs, manages their lifecycles, and secures them.
The crucial concepts you need to troubleshoot effectively.

The `$PATH` Variable

Linux does not search your entire hard drive when you type a command. It only checks a specific, prioritized list of folders. This list is saved in an environment variable called $PATH.

You download a new binary tool (like kubectl or terraform). You type its name in the terminal, but Linux replies with the dreaded command not found. To fix this, you must either move the binary into a folder that is already in your $PATH, or add your custom folder to the $PATH.

# Command: View the list of directories Linux currently checks for commands

echo $PATH

# Output Proof (Directories are separated by colons):
/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin

# Command: Add your custom scripts folder to the PATH for the current session

export PATH="/home/saeed/custom-scripts:$PATH"

How Binaries Work (Shared Libraries)

When you run a command like nginx or python, you are executing a binary file. In Linux, executables are typically formatted as ELF (Executable and Linkable Format) files. However, most binaries are not standalone; they rely on external pieces of code called shared libraries (.so files, similar to .dll files in Windows).

You compile a Go application or migrate a tool into a minimal Docker container (like Alpine Linux). When you run it, it crashes silently or throws a confusing no such file or directory error—even though you can see the binary sitting right there. Usually, the binary isn't missing; its required libraries are missing.

You can use the ldd command to print the shared libraries required by any program.

# Command: Verify exactly which shared libraries Nginx needs to run
ldd /usr/sbin/nginx

# Output Proof:
        linux-vdso.so.1 (0x00007ffe3b1f4000)
        libcrypt.so.1 => /lib/x86_64-linux-gnu/libcrypt.so.1 (0x00007f3a8b411000)
        libpcre.so.3 => /lib/x86_64-linux-gnu/libpcre.so.3 (0x00007f3a8b39c000)
        libssl.so.3 => /lib/x86_64-linux-gnu/libssl.so.3 (0x00007f3a8b2f6000)
        libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f3a8b0cd000)

File Permission

Linux strict permissions dictate exactly who can Read, Write, or eXecute a file. This is often represented by numbers (like 777 or 644).

You are trying to connect to a cloud server via SSH using a private key (.pem file), but the connection is instantly rejected. If you set a .pem file to 777 (meaning everyone on the system can read, write, and execute it), SSH will intentionally reject it for being too exposed. Private keys must be set to 400—meaning absolutely no one except the owner is allowed to read it.

# Command: Attempting to SSH with a bad permission key
ssh -i my-key.pem ubuntu@10.0.0.5

# Output Proof:
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@         WARNING: UNPROTECTED PRIVATE KEY FILE!          @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
Permissions 0777 for 'my-key.pem' are too open.

# Command: Secure the key so only the owner can read it (4), and no one can write (0) or execute (0)

chmod 400 my-key.pem

Processes and Signals

Every running program is a process assigned a unique Process ID (PID). The operating system communicates with and controls these processes using "Signals."

An application is hogging 100% of the CPU and needs to be stopped, or Kubernetes is trying to scale down a pod without dropping active user connections. You can send a SIGTERM (Signal 15), which politely asks the program to save its data and exit cleanly. Alternatively, a SIGKILL (Signal 9) forcefully terminates it at the hardware level, bypassing cleanup and potentially causing data corruption.

# Command: Find the Process ID of an application
pgrep -la nginx

# Output Proof:
1234 nginx: master process
1235 nginx: worker process

# Command: Politely ask the master process to shut down and clean up
kill -15 1234

File Descriptors

Remember concept , "Everything is a File"? That rule has a dark side.

Because Linux treats network connections as files, every time a user connects to your web server, the operating system opens a "File Descriptor" to track that connection. To protect itself from crashing, the Linux kernel sets strict limits on how many file descriptors a single process can open at once.

Your Nginx web server or Node.js application is suddenly dropping user connections during a high-traffic spike. You check the CPU and RAM, and both are perfectly fine. You check the Nginx error logs and see the dreaded worker_connections are not enough or Too many open files error. The operating system has forcibly stopped your app from accepting new traffic.

# Command: Show the maximum number of open file descriptors allowed for this user
ulimit -n

# Output Proof:
1024

# Command: Check how many file descriptors the entire system is currently using
cat /proc/sys/fs/file-nr

# Output Proof:
1205   0   9223372036854775807

Blog Summary

Understanding Linux goes beyond knowing where files live; it’s about understanding how the operating system executes code, secures data, and manages its own limits. This guide breaks down the invisible mechanics of Linux that every DevOps engineer must master to troubleshoot effectively.

You'll learn why binaries silently fail due to missing shared libraries, how to quickly fix "command not found" errors using the $PATH variable, and the strict permission rules required to secure SSH keys. We also cover how to safely manage and terminate running processes using signals, and finally, uncover the hidden trap of File Descriptors—explaining why servers suddenly drop connections with "Too many open files" errors and how to raise those limits.

Beyond the Terminal: My Hunt Through the Linux File System

Sat, 25 Apr 2026 05:07:24 +0000

At first, Linux felt like magic. You open the terminal, type a command… and things just work. No questions asked. But after a while, that “magic” started bothering me.

Because I kept thinking:

Where is all of this actually happening?
How does the system know what to do?

Like… when I connect to the internet, where is that configuration stored?
When a user logs in, where is that data coming from?
When something breaks, where does Linux keep track of it?

These weren’t just random questions anymore. They were pulling me deeper.

So instead of just using Linux… I decided to explore it. Not like a normal user.
But like someone trying to understand what’s going on behind the scenes.

Digging into folders.
Opening system files.
Trying to connect the dots.

And what I discovered?

It completely changed how I look at Linux. Because behind every command…
there’s a system quietly doing a lot more than we realize.

`/etc` - The Place Where System Behavior Lives

The first place I explored was /etc, because I kept hearing that it’s important.

So I went inside:

cd /etc
ls

At first, it just looked like a collection of random files. But the more I opened them, the more I realized something important.

/etc is not just a folder.

It is where Linux stores all configuration that controls system behavior.

When I opened files like:

cat /etc/hosts
cat /etc/passwd

I could literally see how my system is configured.

This made me realize why this folder exists. Instead of hardcoding system behavior deep inside programs, Linux stores configurations as editable files. This means the system becomes flexible you can change behavior without touching the core system.

The problem this solves is huge. Imagine if you had to recompile the OS just to change a setting. That would be impossible for daily use.

But with /etc, Linux separates configuration from execution.

And the insight that hit me was this:
My system is not controlled by hidden magic. It’s controlled by plain text files that I can read and understand.

`/etc/resolv.conf` - How My System Finds Websites

One thing I never really questioned was typing something like:

ping google.com

It just works. But then I stopped and thought:

How does my system even understand what “google.com” is?
Because at the end of the day, computers don’t understand names, they understand IP addresses.

So I decided to dig into it.

I ran:

cat /etc/resolv.conf

And I saw this:

nameserver 10.0.2.3

At first, this confused me.

I was expecting something like 8.8.8.8 (Google DNS).
But this… looked different.

So what is this `10.0.2.3`

It turns out, this is not a public DNS server. It’s a local DNS resolver provided by my virtual machine environment

Since I’m running Linux inside a VM, my system is not directly talking to the internet.

Instead, it works like this:

My Linux system → asks 10.0.2.3 → which then forwards the request to actual DNS servers

So basically, 10.0.2.3 is acting like a middleman.

Why does this file exist?

This file tells the system: “If you don’t understand a domain name, ask this nameserver.”

Without this, the system wouldn’t know where to send DNS queries.

What problem does it solve?

Humans use names like:

But systems use IP addresses. So DNS acts as a translator between human-friendly names and machine-friendly IPs.

And /etc/resolv.conf tells Linux where that translator lives.

What I found interesting

The biggest realization for me was this: My system isn’t always directly using public DNS.
In my case, it’s using a local resolver (10.0.2.3), which then handles everything behind the scenes.

That means even something as simple as opening a website involves:

local system configuration
virtual networking layers
DNS forwarding

Small but powerful insight

Just by changing this file, I can control how my system resolves domains.

Which means I can switch to a faster DNS, block certain domains, & control network behavior.

And honestly… Something as simple as opening a website suddenly felt way deeper than I thought.

Routing Table - How Linux Decides Where Data Goes

Next, I wanted to understand how my system sends data.

So I ran:

ip route

What I saw was something like:

This is the routing table.

At a basic level, this tells the system: “If you don’t know where to send data, send it to this gateway.”

In my case, that gateway is 10.0.2.2.

This exists because your system doesn’t just randomly send data. It needs a structured way to decide where packets go.

The problem it solves is routing confusion. Without this, your system wouldn’t know whether to send data locally or over the internet.

The second line (10.0.2.0/24) tells the system that anything in this local network range can be accessed directly, without going through the gateway.

The insight here was surprising: My system is not just sending data it’s making routing decisions like a mini network controller.

And since I’m using a virtual machine, 10.0.2.2 is actually a virtual gateway, not a physical router.

I always thought networking happens “somewhere outside”. But Linux is actively involved in deciding paths. And in this case, it’s even routing traffic through a virtual layer before it reaches the real network.

`/var/log` - The Silent Observer of Everything

Then I explored logs:

cd /var/log
ls

And opened files like:

cat messages
cat secure

This folder is basically the memory of the system.

Everything that happens:

logins
errors
services
system events

…it all gets recorded here. This exists because systems need a way to track what’s happening internally.
The problem it solves is debugging and monitoring. Without logs, if something breaks, you’re blind.

But what really stood out to me was this:

Linux doesn’t interrupt you with errors all the time.
It quietly writes everything down.

It’s like the system is constantly saying:
“I’ll tell you everything… but only if you come and look.”

`/etc/passwd` & `/etc/shadow` - How Linux Handles Users Securely

To understand users, I ran:

cat /etc/passwd
sudo cat /etc/shadow

What I saw was interesting.

/etc/passwd contains user details.

But passwords are not there.

They’re stored in /etc/shadow, which requires special permissions.

This exists for security reasons. If everything was stored in one place, anyone could read sensitive data.

The problem it solves is protecting user credentials.

But the insight here was deeper:

Linux separates identity from authentication.

/etc/passwd → who you are
/etc/shadow → how you prove it

That separation makes the system more secure by design.

Permissions - Built-In Security, Not an Add-On

When I ran:

ls -l

I saw things like:

-rwxr-xr--

At first, it looked confusing. But then i understood: this is how Linux controls access to every file.

Permissions exist because not every user should have full control over everything.

The problem it solves is unauthorized access and system damage.

But what really stood out was:

Security in Linux is not something extra.
It is built directly into the file system itself.

Every file carries its own rules.

`/proc` - Where I Saw the System Live

This was the most surprising part

I explored:

cd /proc
ls

And then:

cat /proc/cpuinfo
cat /proc/meminfo

What I realized is: These are not real files stored on disk. They are generated in real time by the kernel. This exists so users and programs can access live system data.
The problem it solves is system monitoring.

But the insight here blew my mind:

When I read these files, I’m not reading stored data.
I’m interacting with the system live.

It felt like I was directly talking to Linux.

`/dev` - Where Hardware Becomes Files

Then I explored:

ls /dev

I saw entries for disks, terminals, and devices.

This exists because Linux treats hardware as files.

The problem it solves is complexity. Instead of learning different ways to interact with devices, everything becomes file operations.

And the insight here was simple but powerful: In Linux, everything is unified under one idea: files.

That consistency makes the system elegant.

`/boot` - The Starting Point of Everything

Finally, I checked:

ls /boot

This folder contains the kernel and bootloader files.

This exists because the system needs instructions to start itself. The problem it solves is system initialization.
And the insight: Without /boot, Linux cannot even begin to run.

It’s literally where everything starts.

`/var/log/secure` - Who Tried to Enter My System?

While exploring logs, I went deeper:

So I ran:

sudo tail -f /var/log/secure

And what I saw was honestly eye-opening.

This file logs things like:

login attempts
sudo usage
authentication failures

Basically… anything related to who is trying to access the system.

This exists because systems need to keep track of access.

Who logged in
Who tried and failed
Who used admin privileges

Without something like this, there would be no way to monitor system security.

The problem it solves is pretty serious. If something suspicious happens, like:

repeated failed login attempts
unauthorized access
misuse of sudo

you need a way to trace it. And that’s exactly what this file does. The insight here really changed how I see system logs.

Linux doesn’t constantly warn you about everything.
Instead, it quietly records everything in the background.

And this file becomes like a security audit trail. If someone tries to break into your system… the evidence is already there.

`/dev/null` - The Black Hole of Linux

This one was honestly one of the coolest discoveries.

I started with something simple:

ls /dev/null

Then I tried this:

echo "hello" > /dev/null

And… it disappeared.

No output. Nothing showed up anywhere. That’s when I realized what was happening.

/dev/null is a special file that discards anything written to it. This exists because sometimes, you don’t care about output. Maybe a command is producing too much noise, or you just want to run something silently.

The problem it solves is simple but important. Without /dev/null:

your terminal would get cluttered
scripts would print unnecessary output
things would become harder to manage

The insight here was actually pretty cool.

/dev/null is like a black hole. Anything goes in… and nothing comes out. And suddenly, this made sense:

command > /dev/null 2>&1

This basically tells the system:

Run the command, but don’t show me anything.

And now I finally understood why developers use it so often.

`systemd` - The Hidden Manager Running Everything

Then I got curious about what’s running behind the scenes.

So I explored:

cd /etc/systemd/system
ls

At first, it just looked like a bunch of configuration files.

But then it clicked. This is where system services are defined and managed.

systemd is basically the component that handles:

services
startup processes
background tasks

It’s the reason your system feels “alive” even when you’re not doing anything.

This exists because modern systems need automation. You don’t manually start things like:

networking
logging
system monitoring

Every time you boot your system.

The problem it solves is huge. Without something like systemd you would have to manually start every service after boot . Which is clearly not practical. The insight here was interesting. Even when I’m doing nothing…
Linux is still running multiple services silently in the background. Things just work because systemd is managing everything for me.

`/boot/grub2/grub.cfg` - The First Instructions Linux Follows

This one felt powerful.

I ran:

cat /boot/grub2/grub.cfg

And I was looking at the configuration used by GRUB the bootloader.

This file tells the system:

which kernel to load
how to start Linux

This exists because before Linux even starts, something needs to:

initialize the system
load the kernel
hand over control to the OS

And that’s exactly what the bootloader does.

The problem it solves is fundamental. Without this configuration:
the system wouldn’t even know how to boot. No startup. No OS. Nothing.

The insight here made me realize how critical this file is. This is literally where the system’s startup behavior is defined. If this file gets corrupted Linux won’t even start
And that’s when it hit me… This isn’t just another config file. This is the starting point of everything.

After exploring all these parts, one thing became very clear to me:

Linux is not just an operating system
It is a collection of well-structured systems connected through files

From booting → to users → to networking → to logs

Everything is transparent, accessible, & understandable.
And once you start exploring it like this… you stop guessing and start understanding.

Linux is not hiding anything from you.
Everything is there in files, waiting to be explored.

The moment you stop just using Linux… and start reading it… that’s when things finally start clicking.

Coder Legion - Recent posts tagged linux-commands

Linux Fundamentals for DevOps: Filesystem

/etc — The Control Room (Configurations)

/var — The Growing Data Hub

/home and /root — User Territories

/proc — The Matrix (A Window Into the Kernel)

/dev — Everything is a File

/usr — The Real Home of Installed Software

/opt — Third-Party Add-ons

/tmp — The Throwaway Folder

Blog Summary

Linux Fundamentals For DevOps: Linux core mechanism

The $PATH Variable

How Binaries Work (Shared Libraries)

File Permission

Processes and Signals

File Descriptors

Blog Summary

Beyond the Terminal: My Hunt Through the Linux File System

/etc - The Place Where System Behavior Lives

/etc/resolv.conf - How My System Finds Websites

So what is this 10.0.2.3

Why does this file exist?

What problem does it solve?

What I found interesting

Small but powerful insight

Routing Table - How Linux Decides Where Data Goes

/var/log - The Silent Observer of Everything

/etc/passwd & /etc/shadow - How Linux Handles Users Securely

Permissions - Built-In Security, Not an Add-On

/proc - Where I Saw the System Live

/dev - Where Hardware Becomes Files

/boot - The Starting Point of Everything

/var/log/secure - Who Tried to Enter My System?

/dev/null - The Black Hole of Linux

systemd - The Hidden Manager Running Everything

/boot/grub2/grub.cfg - The First Instructions Linux Follows

`/etc` — The Control Room (Configurations)

`/var` — The Growing Data Hub

`/home` and `/root` — User Territories

`/proc` — The Matrix (A Window Into the Kernel)

`/dev` — Everything is a File

`/usr` — The Real Home of Installed Software

`/opt` — Third-Party Add-ons

`/tmp` — The Throwaway Folder

The `$PATH` Variable

`/etc` - The Place Where System Behavior Lives

`/etc/resolv.conf` - How My System Finds Websites

So what is this `10.0.2.3`

`/var/log` - The Silent Observer of Everything

`/etc/passwd` & `/etc/shadow` - How Linux Handles Users Securely

`/proc` - Where I Saw the System Live

`/dev` - Where Hardware Becomes Files

`/boot` - The Starting Point of Everything

`/var/log/secure` - Who Tried to Enter My System?

`/dev/null` - The Black Hole of Linux

`systemd` - The Hidden Manager Running Everything

`/boot/grub2/grub.cfg` - The First Instructions Linux Follows