Coder Legion - Recent posts tagged scripting

How to Explain a Process with an Animation Explainer

Mon, 26 Oct 2026 09:17:49 +0000

An animation explainer is a short moving picture that makes one idea clear. When the idea is a process, a before and after can carry the whole point: show the starting state, then the ending state, and let the viewer feel the change. Use this shape when the start and end are easy to picture and the middle is short. If the middle is long or branching, show the steps instead.

What matters when you choose an animation explainer?

The starting state and the ending state are both easy to picture.
The change solves a problem the viewer already feels.
The middle fits in a few visible moves.
The viewer can hold both states in mind on one pass.
The ending state removes a real cost, not just tidies it.

If two or three of these are weak, the format is probably wrong. That is not a failure of the idea; it is a sign to choose a different shape.

The decision is not about style; it is about what the viewer needs first. If the change is the news, lead with the change. If the steps are the news, lead with the steps and let the ending state arrive last.

How do you judge each criterion?

Start with the two pictures, not the steps. If you cannot sketch the before and after in a few seconds, the viewer will not hold them either. A messy shared drive and a tidy folder system works because both are visible, while "better data hygiene" is not. If the pictures need the words to make sense, the pictures are not doing the work.

The change has to solve a problem the viewer already feels. If the after state only saves time for the team that runs the process, the before and after will feel like an internal memo dressed up as a story. Tie the ending state to something the viewer loses today: lost files, repeated questions, a task that takes too long, or a handoff that fails. The before picture should make that loss visible in one glance.

Count the moves between the two pictures, not the minutes. If the middle needs more than a handful of actions, the before and after will skip the part that actually matters. In the shared drive example, rename, sort, and remove duplicates is short; adding permissions, naming rules, and training is not. When the middle grows, switch to a step-by-step explainer video and keep the before and after as the first and last frames.

The viewer should see both states without scrolling back or replaying. That means one screen, or one clear transition, not a long tour of details that only make sense in order. If you need a second viewing to compare, the change is too large for this format. A single animated video can still show detail, but only after the main change is locked in and the viewer knows what to look for.

The after state must look like relief, not like a different kind of work. If the new process only moves the pain somewhere else, the before and after will read as a trade, not a win. Name the cost that disappears, and make sure it disappears in the picture, not only in the voiceover. Show the moment the old cost stops, and let the new state stay on screen long enough to feel earned.

The trade-off is simple: clarity now versus completeness later. A before and after buys attention and care; it does not buy a full procedure. If you need the full procedure, plan a second piece.

When you judge the criteria, judge them in this order: pictures first, then pain, then middle, then memory, then relief. If the first fails, stop; the rest do not matter. This order saves time and protects the idea.

Which red flags show it is the wrong choice?

Some processes fight this format. The signs usually show up while you are still planning, before anyone draws a frame or writes a line. If two or more of these are true, plan a step-by-step explainer video instead and keep the before and after for a single closing shot.

The before and after look almost the same on screen.
The process has branches, exceptions, or approvals that carry the real point.
The viewer needs numbers, labels, or a legend to see the change.
The after state needs a long explanation of why it is better.
The middle contains a decision that changes the outcome.
The same process looks different for different people.

These are not small flaws; they are format mismatches. A before and after in an animation explainer works when the change is the story and the steps are short enough to trust. When the steps or the rules are the story, show them in order and let the ending state arrive last. The viewer will still feel the improvement, but they will also understand how to reach it.

Do not force a process into this shape because the pictures look good. The format is a promise that the change is visible and short. Break that promise and the viewer stops trusting the next animation explainer you make.

If only one red flag is true, you may still use the format, but name the limit in the script. Honesty about the limit keeps the piece useful. Hiding it turns a small mismatch into a broken promise.

What should you ask before you commit?

These questions force the change into words before it becomes pictures. If an answer stays vague, the animation will stay vague too.

What exactly changes between the first frame and the last frame?
Can a newcomer name that change in one sentence?
Which step in the middle carries the risk, cost, or delay?
What would make the after state feel unearned?
How will you show the change with pictures before you add words?
Who in the audience will not feel the problem in the before state?

If the last question has no clear answer, the before picture is probably too generic. Fix the before state first; the after state depends on it.

Answer them in plain words before you sketch. If an answer needs a diagram to make sense, the picture is not ready yet. The questions are a filter, not a script.

Frequently asked questions

Can a before and after replace the whole process?

It can replace the overview, not the training. Use it to make someone care about the change, then use a step-by-step format for the actions. If you try to teach every step inside the before and after, the middle will blur and the ending will feel unearned. The viewer remembers the feeling of the change, not the procedure.

A good handoff is to end with one clear next step and a place to find the full steps. That keeps the animation explainer focused on the change while still respecting the viewer who wants to act. The feeling comes first; the procedure follows.

What if the process has more than one good outcome?

Pick the outcome that removes the pain the viewer already knows. Showing two or three endings turns the piece into a comparison chart, which is a different job with different pacing. If the outcomes depend on context, say so in one line and keep the main change simple. You can always end with a short note that other paths exist.

When context matters, name the condition in the voiceover and keep the picture on the main case. The viewer can hold one exception in mind if it is stated plainly. Two exceptions usually need a different format.

How long should the middle part be?

Short enough that the viewer does not lose the thread. If you need to pause, rewind, or add a label to keep up, the middle is too long for this shape. Cut the middle to the moves that change the picture, and move the rest into a later section or a separate explainer video script. The middle is not the place to be complete; it is the place to be believable.

A believable middle has fewer moves than a complete one. When in doubt, cut the middle by half and see if the change still makes sense. Most middles survive the cut better than their authors expect.

Try sketching the before and after on paper before you write a script. If both pictures are clear and the middle fits in a few moves, you have a strong shape for an animation explainer. If not, switch to steps and save the before and after for the final frame.

My ERR trap stayed silent until I added one letter to set -euo pipefail

Tue, 22 Sep 2026 13:00:00 +0000

A nine-line script, set -euo pipefail on line 2 and an ERR trap on line 3 whose only job is to print the failing command and its line number. A helper function then runs mkdir /proc/export, a directory that cannot be created. On bash 5.3.9 it printed mkdir's own complaint and exited 1. The trap said nothing: no command, no line, the two facts it was registered to report.

By default the ERR trap is not inherited by functions, command substitutions or subshells. Errexit killed the script from inside the function; the trap, registered at the top level, never saw the failure. One letter fixes it:

set -Eeuo pipefail
trap 'echo "✗ failed: ${BASH_COMMAND} (line ${LINENO})" >&2' ERR

-E (errtrace) makes the trap follow the script into functions. Same run, same mkdir error, now followed by ✗ failed: mkdir /proc/export (line 5).

Two more silent spots from the same session. local out=$(false) inside a function carried on to the next line and the script exited 0, because local succeeded and its status is the one bash checks. Declare on one line, assign on the next. And in an EXIT handler, capture $? on the first line: moved below an rm -f, it reports rm's zero, and a script ending in exit 3 exits 0.

Where errexit goes silent, nounset defaults and the assembled skeleton with both traps are in the strict-mode bash script template, checked against bash 5.3.

The Alert Never Fired Because the Loop Skipped the Last Line of the File

Fri, 19 Jun 2026 17:58:59 +0000

We kept a plaintext file of hostnames, one per line, and a monitoring script read the file and pinged each host every five minutes. When a host failed to respond, the script sent an email alert. The system had been running for months and it worked — we had caught three actual outages with it, which gave us real confidence in the setup.

app-07 was added to the list on a Thursday afternoon. The engineer who added it was using VS Code on a Mac, and VS Code by default does not add a trailing newline to a file when you append to it using certain editing workflows. The file had ended in a newline before the edit. After the edit, the last line — app-07 — had no trailing newline.

app-07 went down the following Sunday afternoon at 2:17pm. The monitoring script ran at 2:20, 2:25, 2:30, all the way through the evening. No alert ever fired. The on-call engineer found out at 8pm when a client emailed. The system had been down for almost six hours.

When I looked at the script, the bug was immediately obvious once I knew what to look for. But I had written that script, I had tested it, and I had been looking at the monitoring confirmation emails for months without ever noticing. The confirmation email listed the hosts it checked. app-07 was never in the list. I had been reading those emails without actually counting the hosts. I just scanned for the OK lines and moved on.

Why read drops the last line

read returns a success exit status when it reads a line and finds the newline that terminates it. When the file does not end in a newline, read still populates the variable with the final line's content, but it returns a non-zero (failure) exit status because it hit end-of-file before finding a terminator. A while read host loop checks the return status to decide whether to execute the loop body. On the final, newline-less line, read puts app-07 into host and then returns failure. The while loop sees failure and exits without running the body. The content is there. The variable is populated. The loop throws it away.

This behavior is documented in the POSIX spec for read. It is not a bash quirk. Any POSIX shell handles the missing-final-newline case this way. A plain while read line loop is incorrect for any file you do not personally control the formatting of, which in practice means nearly any file.

The fix is one extra clause:

#!/bin/bash

# Script: check-hosts.sh

# Purpose: ping every host in a file, including a newline-less final line

# Usage: ./check-hosts.sh hosts.txt

set -euo pipefail

CHECK="✓"
CROSS="✗"
HOST_FILE="${1:?Usage: check-hosts.sh <host-file>}"

while IFS= read -r host || [[ -n "$host" ]]; do
  [[ -z "$host" || "$host" == \#* ]] && continue
  if ping -c1 -W2 "$host" >/dev/null 2>&1; then
    echo "$CHECK up:   $host"
  else
    echo "$CROSS down: $host"
  fi
done < "$HOST_FILE"

|| [[ -n "$host" ]] says: if read returned failure but the variable is non-empty, run the loop body anyway. That is precisely the leftover-final-line case. read failed because it hit end-of-file, but it populated host with app-07 before returning. The || catches it. app-07 gets pinged.

What IFS= and -r actually do

You see while IFS= read -r line written in every correct read-loop example, and it is worth being specific about what each piece prevents because both have their own failure mode.

IFS= sets the field separator to empty for the duration of the read command. Without it, read strips leading and trailing whitespace from each line. A hostname like app-07 (with leading spaces, which some editors produce) becomes app-07, which might be correct. An indented config value, a Python-style YAML string, a log line that starts with spaces for alignment — all of these are silently modified. Setting IFS= tells read to take the line exactly as it appears.

-r prevents read from interpreting backslash sequences. Without -r, a line like C:\temp\logs has its backslashes consumed as escape characters and arrives as C:templogs. This matters less for hostname files and enormously for any script that processes Windows paths, config files that use backslash as a line-continuation character, or log files from mixed-OS environments. The -r flag is essentially free protection; there is no reason not to include it.

A bare read line without either flag silently mangles both whitespace and backslashes. The script works correctly on clean input and produces wrong output on input with edge cases. The wrong output does not produce an error. You find out when the data that mattered was the indented or backslash-containing kind.

The subshell trap that kills your counters

This is the one that is most likely to make you question your sanity:


# This looks correct. It is not.

fails=0
cat hosts.txt | while IFS= read -r host; do
  ping -c1 -W2 "$host" >/dev/null 2>&1 || ((fails++))
done
echo "Total failures: $fails"   # Always prints 0

The pipe creates a subshell for the right side. The while loop runs inside that subshell. fails increments correctly inside the subshell. When the subshell exits, the parent shell's fails is still 0, because the increment happened in a different process. The parent echo sees the original value.

This catches people because the loop body itself works — the pings happen, the increment logic is correct — but any state the loop was supposed to accumulate for later use is silently discarded. I spent forty minutes on a version of this problem before I remembered that pipes create subshells. It is the kind of thing that feels like a bash bug until you understand that it is behaving exactly as documented.

The fix is to redirect the file into the loop instead of piping into it:

fails=0
while IFS= read -r host || [[ -n "$host" ]]; do
  ping -c1 -W2 "$host" >/dev/null 2>&1 || ((fails++))
done < hosts.txt
echo "Total failures: $fails"   # Now correct

done < hosts.txt feeds the file to the loop's stdin without a pipe. The loop runs in the current shell. fails accumulates in the current shell. The echo sees the real count.

Why for loop is wrong for this


# Never do this — iterates words, not lines

for line in $(cat hosts.txt); do
  ping -c1 "$line"
done

$(cat hosts.txt) is command substitution. Bash captures the text output and word-splits it on IFS — spaces, tabs, newlines. For a file with one hostname per line and no spaces in the hostnames, this accidentally produces the right behavior. For any file with spaces — log lines, config values, paths with spaces, anything a non-developer might have generated — it splits lines into fragments and each fragment becomes a loop iteration.

There is no version of for line in $(cat file) that is correct for reading lines. The right tool is always while IFS= read -r line || [[ -n "$line" ]]; do ... done < file. The for loop is the right tool for iterating a known list that you control directly, not for reading file content.

The monitoring system, after the fix

After the app-07 incident we made three changes. The obvious one was fixing the read loop with the || [[ -n "$host" ]] guard. The second was adding a sanity check at the top of the script that counted the lines in the hosts file and compared it to the number of hosts the loop actually processed — a mismatch meant the file was malformed or something else was wrong. The third was adding a nightly email that included the count of hosts checked, not just the status of each one, so a future addition to the file that somehow got lost would show up as "expected 12 hosts, checked 11."

The confirmation email count was something I should have had from the start. If I had been looking at "checked 11/12 hosts" instead of a list of OK lines, I would have noticed app-07 missing on the first night. The monitoring was working. The observability of the monitoring was not.

Full version with CSV-field parsing, comment-skipping, and the subshell-safe redirect form: https://bashsnippets.xyz/snippets/bash-read-file-line-by-line

To iterate a list of files rather than a file's contents, reach for a for loop instead, and wrap anything that acts on what it reads in set -euo pipefail. More at https://bashsnippets.xyz

Originally published at bashsnippets.xyz/snippets/bash-read-file-line-by-line

A For Loop Skipped Every File With a Space and Called the Backup a Success

Thu, 18 Jun 2026 19:28:03 +0000

The nightly backup looped over for f in $(ls /data/exports) and copied each file to a backup volume. It exited clean every night. For three weeks, green exit codes, no errors, nothing in the logs to suggest anything was wrong. The backup script had been written by someone who left the company six months before I started, and it had never been tested against files with spaces in their names because the original export directory only ever had files like Q3.xlsx and report-final.xlsx.

Then someone generated Q3 final.xlsx.

It took three weeks because that file was generated once a quarter and the next time someone needed it was a quarter later. The person who needed it was the CFO. The CFO does not particularly enjoy being told that the backup system that was supposed to protect the quarterly export has been silently failing for an indeterminate period of time and we are not sure which other files it missed.

I know the specific backup had been running for three weeks because that was the date the file appeared in the exports directory. Every night since then, the loop had been splitting Q3 final.xlsx into two items — Q3 and final.xlsx — trying to copy two paths that did not exist, logging two harmless "no such file or directory" lines that nobody read, and moving on. Every file without a space in its name backed up fine. The script looked correct because most of the time it was.

Why $(ls) word-splits

When you write for f in $(ls /data/exports), bash runs ls, captures its text output, and splits it on IFS — the internal field separator, which defaults to spaces, tabs, and newlines. Filenames are just text in the output of ls. A file named Q3 final.xlsx is one filename, but ls outputs it as the string Q3 final.xlsx, and bash splits that string on the space into two separate items before the loop ever starts.

This is not a bug in ls. This is not a bug in bash. This is exactly what $( ) does to any command's output — it captures text and bash processes it as text. The problem is that filenames are not reliably text-safe; they can contain any character except null and the path separator. Spaces are common. Tabs are less common but legal. Newlines are technically legal. Treating command output as a filename list breaks the moment any of those show up.

The fix is to stop treating command output as a filename list and let bash build the list from the filesystem directly:

#!/bin/bash
# Script: backup-exports.sh
# Purpose: copy export files without losing ones with spaces in their names
# Usage: ./backup-exports.sh
set -euo pipefail

CHECK="✓"
CROSS="✗"
SRC_DIR="/data/exports"
DEST_DIR="/backup/exports"
shopt -s nullglob   # empty glob expands to nothing, not the literal pattern

for f in "$SRC_DIR"/*.xlsx; do
  if cp -- "$f" "$DEST_DIR/"; then
    echo "$CHECK backed up: $(basename "$f")"
  else
    echo "$CROSS failed: $(basename "$f")"
  fi
done

for f in "$SRC_DIR"/*.xlsx asks bash to expand the glob. Bash talks directly to the filesystem and gets back a properly-separated list of matching paths. No command output, no text splitting, no ambiguity about what the separator is. A file named Q3 final.xlsx stays one item because it was never turned into text and split back apart. The -- before "$f" tells cp to stop reading flags, so a filename that starts with a dash does not get interpreted as a cp option.

The second half: quoting on use

The glob fixes the loop header. Quoting fixes the point of use. These are separate.

for f in "$SRC_DIR"/*.xlsx; do
  cp $f "$DEST_DIR/"    # WRONG — $f re-splits here
  cp "$f" "$DEST_DIR/"  # RIGHT — the quotes prevent the split
done

Even with a correct glob, an unquoted $f in the cp command re-splits on spaces. Bash has already expanded the variable to Q3 final.xlsx, but when you use it unquoted, the shell processes word splitting again on that value and cp receives two arguments: Q3 and final.xlsx. The quotes around "$f" tell bash to pass the entire value as a single argument. The rule is: glob over parse to build the list, quote on use to keep each item intact.

I have seen this mistake repeated in scripts at three different companies. In each case the scripts had been running for months or years and the problem was invisible because the majority of files had no spaces. The ones that did — client names, quarterly reports, anything a non-technical person named — were being silently skipped or mishandled.

Ranges and counters: where the brace trap hides

n=5

# This does NOT produce a range — it produces the literal string {1..5}
for i in {1..$n}; do echo "attempt $i"; done

# This works — C-style, evaluates variables at runtime
for ((i = 1; i <= n; i++)); do echo "attempt $i of $n"; done

Brace expansion happens before variable expansion in bash's order of operations. By the time $n is replaced with its value, the brace expansion step has already passed and {1..$n} is just a string. This is the kind of thing that fails silently in a test environment where the count is small and hardcoded, and causes weird output in production where it is a variable.

The C-style loop is the correct form any time the bound is a variable. It evaluates at runtime and handles arithmetic naturally. If the count is genuinely fixed at write time, {1..10} works fine. If it is ever going to be a variable, use for (( )).

Arrays: the original bug wearing a different hat

servers=("web-01" "db primary" "cache-02")

# WRONG — word-splits "db primary" into two iterations
for s in ${servers[@]}; do
  ping -c1 "$s"
done

# RIGHT — quotes keep each element intact
for s in "${servers[@]}"; do
  ping -c1 "$s"
done

"${servers[@]}" with the quotes and [@] is the form that preserves each element as a single item regardless of what is in it. Without the quotes, bash word-splits the array expansion and db primary becomes two separate loop iterations — neither of which is a real hostname. This is exactly the same word-splitting mechanism as the $(ls) problem, just manifesting in arrays instead of command output.

Once you internalize that word-splitting happens wherever an unquoted variable or expansion appears, the rule becomes one rule instead of several: always quote expansions. The specific context changes; the mechanism does not.

The nullglob case

shopt -s nullglob
for f in /data/exports/*.xlsx; do
  echo "$f"
done

Without shopt -s nullglob, if no .xlsx files exist, the glob *.xlsx does not expand — it stays as the literal string *.xlsx. The loop runs once with f set to the literal string /data/exports/*.xlsx. Your script then tries to process a file with that exact path, which does not exist, and produces an error or silently does nothing depending on what you do with it.

Setting nullglob tells bash to expand a non-matching glob to nothing (an empty list), so the loop simply does not run. This is almost always the right behavior when you are iterating files that might not exist.

What happened to the Q3 export

We recovered it from the CFO's local machine, where she had downloaded it before the backup was supposed to preserve it. The fix to the script took four minutes. The conversation about why the backup system had been failing silently for three weeks took longer. The monitoring that we added afterwards — a nightly check that the backup directory has at least as many files as the source directory — took another twenty minutes.

The monitoring should have been there from the start. So should the glob. So should the set -euo pipefail that would have made the copy failures loud instead of silent. These are things you add before something breaks, and the only reason to know you need them is to have seen, or caused, or read about what happens when they are missing.

Full examples with the safe glob, counter, C-style loop, array form, and nullglob guard: https://bashsnippets.xyz/snippets/bash-for-loop-examples

For reading a file's lines one at a time, a for loop is the wrong tool — use while IFS= read -r — and wrap any loop that touches real files in set -euo pipefail. The rest is at https://bashsnippets.xyz

Check out my CoderLegion profile

Thu, 07 May 2026 13:09:31 +0000

Points: 101 | Badges: 4 | Followers: 0 | Posts: 0

View my full profile: https://coderlegion.com/user/Emmanuel Cortes

A Soft Introduction to EasyShell

Tue, 05 May 2026 02:31:45 +0000

"Can it be simpler?" - A recurring thought I've had while working on build automation.

EasyShell (command: easy) is a minimal shell scripting language designed from the ground up for simplicity and cross-platform build/process automation. If you’ve ever felt that PowerShell’s syntax is powerful but easy to forget, or that traditional shell scripts get messy across operating systems, EasyShell aims to strike a cleaner balance.

It's implemented in C# and runs on .NET, giving you natural access to the entire .NET ecosystem while keeping the scripting experience lightweight and approachable.

Design Principles

One command per line (shell-like flow)
Very small set of core concepts
Strongly-typed global variables
Seamless .NET interop via reflection
Expressions in parentheses for inline evaluation
Easy to read, write, and remember

Version 0.1.0 is now available — functional for everyday automation tasks.

Core Language Features

Variables

All variables are declared with an explicit type and live in global scope:

INTVAR Count 42
BOOLVAR Debug TRUE
DOUBLEVAR Ratio 3.14
STRINGVAR Message "Hello from EasyShell"
HANDLEVAR Now System.DateTime.Now   # stores object instances

Variable names are case-insensitive
Reference with $VarName
Re-assign with $VarName = value (or expressions)

Expressions (The Power Feature)

Wrap sub-commands in parentheses to evaluate them inline:

STRINGVAR Formatted (System.String.Format "Count = {0}" $Count)
BOOLVAR IsBig (> $Count 100)
STRINGVAR Today (format "{0:yyyy-MM-dd}" (GetDate))

Calling .NET

Static members (fully qualified):

System.Console.WriteLine "Hello World!"
System.IO.File.WriteAllText "output.txt" $Message

Instance methods via HANDLE:

HANDLEVAR Now System.DateTime.Now
STRINGVAR Stamp (CALL $Now ToString)

Control Flow

IF (== $Count 42)
    print "The answer!"
ELSEIF (> $Count 50)
    print "Too high"
ELSE
    print "Something else"
END

INTVAR I 0
WHILE (< $I 5)
    System.Console.WriteLine $I
    $I = (+ $I 1)   # arithmetic helpers coming soon
END

Functions

Simple named blocks (no parameters — use globals):

FUNC BuildProject
    print "Building..."
    dotnet build
END

CALL BuildProject

External Commands

Any command that isn’t a keyword or .NET call falls back to executing an external process (with stdout capture):

git status
dotnet publish ...

Real-World Example

Check out the full build script used by the project itself:
BuildEasyShell.easy

It demonstrates:

Path handling and folder creation
Running dotnet publish
Conditional logic for OS/architecture
Creating a dated zip package

This script is self-contained and shows how EasyShell can manage non-trivial automation tasks.

Quick Start

# REPL mode
easy

# Run a script
easy path/to/script.easy

# Help
easy --help

A dedicated VS Code syntax highlighting extension is available: Methodox.easyshell

Who Is It For?

EasyShell is ideal for:

Build automation scripts
Simple CI/CD glue
Daily developer tooling
Anyone who wants .NET power without heavy dependency

It's intentionally minimal — no advanced Bash-style features, just the essentials done in a way that’s hard to forget.

Give it a try and let me know what you think!

DevLog 20251214: Yet Another Shell Script – Keeping Build Automation Simple with EasyShell

Tue, 05 May 2026 02:23:48 +0000

As we roll out more tools, both internal and public, there has been a growing need for easy-to-maintain shell scripting and build automation. Build automation is complex enough without all the syntax burden - my favorite is to use PowerShell, but often it feels like wrestling with the tool itself. PowerShell is powerful, sure, but its depth often means hunting down syntax I know I've used a dozen times before. Bash has its quirks across platforms, and honestly I don't know good ways of using it outside MinGW, Git Bash, or WSL - none of which felt quite "right".

I wanted something lighter: a shell scripting tool that's easy to pick up, cross-platform, and hard to forget once you start using it.

So I made EasyShell (or just easy): a minimal shell scripting language implemented in C# that focuses on process automation with a very small set of concepts.

Motivation and Design

EasyShell is for "lazy" developers (like me) who just want to automate common tasks - calling .NET APIs, running external tools, handling simple logic - without a steep syntax curve or platform-specific headaches. It combines a shell-like one-command-per-line flow with strong typing and easy .NET interop. No advanced Bash-style wizardry; just straightforward scripting.

It could also serve as a nice companion alongside other tools in the long-term Divooka ecosystem.

Quick Taste of the Syntax

Files typically use the .easy extension, with syntax highlighting available via the VS Code extension.

Here's a small sample from the repo:

# Comments start with #
HANDLEVAR NOW System.DateTime.Now

# Instance method via CALL
STRINGVAR DATE (CALL $NOW ToString)

# Static .NET call with expression
STRINGVAR VALUE (System.String.Format "Current time: {0}" $DATE)

STRINGVAR PATH "C:/temp/greeting.txt"
System.IO.File.WriteAllText $PATH $VALUE

Variables are declared with their type (INTVAR, STRINGVAR, BOOLVAR, DOUBLEVAR, HANDLEVAR), names are case-insensitive, and everything lives in global scope for simplicity. Expressions are wrapped in parentheses for inline evaluation, like (== $X 10) or sub-commands.

Control flow sticks to the basics:

IF ... ELSEIF ... ELSE ... END
WHILE ... END
Simple FUNC ... END / CALL (globals make "returns" easy via variables)

You can call external executables naturally, and non-built-in commands fall through to process execution (stdout capture included).

Early Days

This is version 0.1.0 territory - functional for everyday build tasks, with room to grow (arithmetic helpers, more examples, etc.). You can run it in REPL mode (easy) or point it at a script. Builds and packages are straightforward with the included scripts.

If you've ever wished for something between a full PowerShell script and a quick batch file that "just works" across machines, maybe give EasyShell a try. The GitHub repo has the full language overview, examples, and build details: MethodoxTech/EasyShell.

Scripting done differently? (python, shell script, JS)

Tue, 14 Apr 2026 07:21:38 +0000

Most of the folks that have done scripting, already have this fixed idea in their heads, thats scripting means JS.

That was the case for me as well till the day someone eventually asked when we would add pre and post requests in Voiden. Our instinct was to say "ok, its coming soon" and then build it quickly. But then we started thinking about all these hidden assumptions around scripting. So we took a bit more time and shipped something nice: scripting as a true runtime layer that:

Supports multiple languages, so you can use the one that fits your stack.
Runs on real interpreters, not a limited sandbox.
You can import packages and reuse your existing logic.
Workflows are stateful, letting you chain requests and maintain context across them.

Already works with JavaScript, Python and shell. Tomorrow, more languages will follow.

The result is something we deeply care about: orchestration-level automation right inside your API client.

check here: https://voiden.md/download
repo: https://github.com/VoidenHQ/voiden

Let's check out the different methods to check PowerShell Version on Windows 11 with clear steps.

Fri, 03 Oct 2025 04:59:16 +0000

How to Check PowerShell Version on Windows 11?

Check PowerShell Version on Windows 11: Windows PowerShell is a command-line shell and scripting language built on top of the .NET Framework. Unlike the Command Prompt, PowerShell allows users to automate tasks, manage system configurations, and control Windows more effectively. The Automation in Windows PowerShell is extensive, supporting scripts, Loops, Functions, and Modules. Microsoft initially introduced it as Monad in its beta phase, which later became Windows PowerShell 1.0 on its 2006 official launch.

Why Should you Check your PowerShell Version on Windows 11?

Different PowerShell Versions support different cmdlets, exhibit scripting capabilities, and provide security updates. For example, Windows 11 is pre-installed with PowerShell 5.1, which is compatible only with Windows. All of its cmdlets, modules, and scripts are designed for the Windows Ecosystem, whereas the latest Windows PowerShell Version 7 and above have cross-platform ability, and it allows users to run PowerShell 7 and above on other operating systems such as Linux and macOS.

Additionally, PowerShell Version 5 receives only legacy security updates, whereas PowerShell 7 receives frequent security patches, performance improvements, and bug fixes. Each version has its own advantages and disadvantages, and in this article, let's discuss them in detail. Let's get started.

Different Methods to Check PowerShell Version on Windows 11

To determine which version of Windows PowerShell you are using, you can either use Windows PowerShell or you can use outside PowerShell methods.

Check PowerShell Version using PowerShell Commands
Get PowerShell Version outside Windows PowerShell
Easily Check Version on PowerShell 7

How to Determine PowerShell Version using PowerShell Commands?

In PowerShell, you can use various commands to get the PowerShell Version. The following are the steps.
Firstly, let's open Windows PowerShell using the Run Command. We can use the keyboard shortcut Win Key + R.
In the Run, execute the following command to open Windows PowerShell. powershell
Open PowerShell using the Run Command
The Run command will open PowerShell on Windows 11.

In PowerShell, we can use the following commands to determine PowerShell Version on Windows 11.

1 Using $PSVersionTable

Execute the command $PSVersionTable in Windows PowerShell.

Note: If you only want to see the PowerShell version, then you can use the following command in the PowerShell $PSVersionTable.PSVersion. This command will precisely provide the determined PowerShell Version and related information. Kindly refer to the following image.

With the above command, only the essential information, such as Major Version, Minor Version, Build, and Revision Number, will be displayed.

2 Get PowerShell Version using the Get-Host Command

The Get-Host Command on Windows PowerShell provides information about the Console Host. The information is not related to the detailed PowerShell Version, however, we will get the PowerShell Version here.

To get the PowerShell Version using the Get-Host command, please execute it in PowerShell.
Execute the Get-Host Command in PowerShell

The output will have the following information. Name, Version, InstanceID, UI, CurrentCulture, CurrentUICulture, Private Data, Debugger Enabled, IsRunspacePushed, Runspace. Here, the version determines PowerShell Version.

3 Determine PowerShell Version Directly using $host.version

This command provides straightforward output similar to $PSVersionTable.PSVersion.
Execute the following command in PowerShell. $host.version
The output will contain the following information. Major Build Number, Minor Build Number, Build Version, and Revision Version.
I am currently running Windows PowerShell 5.1.26100.6682.

How to Determine PowerShell Version Outside of PowerShell?

Alright, the above methods explain how to check PowerShell Version using Windows PowerShell itself. Here, we can check how to check the Windows PowerShell Version outside of PowerShell, such as the Command Prompt and Registry Editor.

Get PowerShell Version using Command Prompt:

Go to the Run and open Command Prompt using the following command. cmd
In the Command Prompt, kindly execute the command powershell -Command "$PSVersionTable.PSVersion"
Now, the Command Prompt will display the PowerShell Version. In my case, I am on PowerShell 5.1.26100.6682.

How to Check Windows PowerShell Version on Windows 11 using Registry Editor?

To open the registry editor on Windows 11, we will use the Run Command.
In the Run, execute the command regedit
Open Registry Editor using the Run Command
The User Account Control will confirm and open Registry Editor.
Navigate to the following path in the Registry Editor. HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\PowerShell\3\PowerShellEngine
Once you navigate to the above path, you can find the file "PowerShellVersion". You can view the version next to it.
You can also open it to view the PowerShell Version. That's it.

How to check the Version in the latest PowerShell 7?

Windows PowerShell 7 is the modern, cross-platform form of PowerShell. Unlike the legacy PowerShell 5, which is built into Windows 11, we have to install PowerShell 7 separately. There are different methods available to install PowerShell 7 on Windows 11, from the Microsoft Store to Winget, MSI, and more. Here, we will explain how to check the PowerShell 7 version.

Install PowerShell 7 using WinGet on Windows 11

Open Command Prompt, and run the following command. winget search powershell
Winget will search the Microsoft Community Repository and Microsoft Store and provide the results.
From the results, you can locate the official stable build and install it using the following command. At the time of this article, the latest version of PowerShell is 7.5.3.0. It is the stable and official release. Kindly note down its ID. The ID is Microsoft.PowerShell
Locate official and stable PowerShell 7 and note down its ID
Next, we will install PowerShell 7 using the following command in the Command Prompt. winget install --id Microsoft.PowerShell
This command will install PowerShell 7 from the official GitHub repository. The process will take some time to complete. Kindly be patient.
Once the package is successfully installed, you can start using Windows PowerShell 7. You can explicitly access it via the Start menu or by using the command pwsh

Note: You can also download and install Windows PowerShell 7 using the MSI installer from the official GitHub Repository.

Determine PowerShell 7 version using pwsh command

The following command will work only on the latest PowerShell 7 version.
In PowerShell 7, execute the following command. pwsh --version
PowerShell will display the version now.

Note: In PowerShell 7, other legacy methods such as Get-Host, PSVersionTable will work flawlessly. Additonally, you can get PowerShell 7 version using the following command as well. pwsh -v

Take Away

Whether you are using Legacy PowerShell 5 or the Modern PowerShell 7, the above article covers different methods and detailed steps to check the PowerShell Version on all variations. We hope you are satisfied with our article on How to Check PowerShell Version on Windows 11, and if you have any queries, kindly let us know in the Comments.

For more interesting articles, stay tuned to Winsides.com. Happy Scripting! Peace out!

DevLog 20250724: C# Scripting, Pure 2, Updated + Some Recap

Fri, 25 Jul 2025 01:39:40 +0000

Intro

Pure has been our in-house C# scripting solution for a few years now.
After experimenting with the well-known CS-Script and some other solutions I can no longer recall, I began experimenting with Roslyn in .NET around 2022/2023, and it's been working well ever since.

Usage

Below is a basic Pure script and its execution:

using static System.Console;

var sum = File.ReadLines("Data.csv")
    .Skip(1)
    .Select(l => l.Split(','))
    .Sum(parts => double.Parse(parts[1]));
WriteLine(sum);

Recap

Version 0.0.3

The first version of Pure was just a simple wrapper around Roslyn. Later, we expanded it to support syntax like Import(MathNet.Numerics). It wasn't until we added native libraries like CSV, ODBC, and CentralSnippets that things started to get interesting.

Motivation

Below is an old table I compiled when first drafting the design document for Pure (1), with some updates to mention newer solutions:

What's Special

Pure 2 is a lightweight C# scripting platform with modern C# 12 syntax and easy NuGet import support. Pure is based on Roslyn. In addition to the base .NET runtime, Pure offers additional libraries for handling common tasks, along with a handy scripting interface (REPL and Notebook) and a place where people can share their snippets.

Pure 2 is conceptually similar to Pure but architecturally very different - it directly builds upon Parcel NExT's Ama runtime (the same runtime used by Divooka). Pure is designed for very short snippets and getting everyday tasks done faster.

Pure 2 uses C# 12 and is similar to top-level statements in C#, with the following features:

You can define types and functions anywhere in the middle of the script.
Pure has its own package import behavior (default namespace usage and package lookup rules).
Default global scope math functions, and default using for System.IO and System.Linq.
Top-level function and variable definitions, alongside classes and other common C# constructs.
Simplifies common tasks even further.
Macros to support text-based parsing, file inclusion, and module imports.

Features

Single-word standard package names. One-command NuGet package import and on-the-fly package invocation (through Import).
Full compatibility with Divooka packages and Divooka graphs.
Pure uses PUREPATH to search for scripts when using Include and when executing script files directly from the command line as the first argument.
Pure is very lightweight and will always be a thin layer on top of Roslyn/.NET runtime.
CLI REPL; GUI Notebook.

Basic Syntax

Certain macros/syntax are provided to implement language-level scripting constructs:

Summary

Be aware that adopting a scripting system - any scripting system - comes with significant risks. It's wise to plan carefully and pick the best tools for the task at hand.

References

Download page for Pure 2 on Methodox Itch.io
Official Page for Pure 2 on Methodox Wiki

Automating With Python: A Developer’s Guide

Tue, 14 May 2024 22:30:57 +0000

In this guide, we'll explore Automation with Python. Automation is key in today's fast-paced world, saving time and effort across various tasks. Python, which is a top choice for Automation. We'll cover its essential libraries along with examples for practical uses. By the end, you'll be ready to automate tasks efficiently using Python.

Understanding Automation

What is automation

Automation is the use of technology to perform different tasks with minimal human intervention. It involves the implementation of systems or processes that can operate automatically, reducing the need for manual labor and increasing efficiency.

Why Automation is Important and Required:

Efficiency: Streamlines processes, saving time and effort.
Accuracy: Minimizes human error, ensuring precision.
Cost Reduction: Lowers labor costs, increases productivity.
Scalability: Adapts easily to changing demands.
Productivity: Frees up time for strategic work.
Data Analysis: Provides valuable insights for decision-making.

Introduction of Python in Automation

Python is a powerful language for automation tasks due to its simplicity, versatility, and extensive libraries. You can also visit Python official site for more information.

Python Modules for Automation

Given below are the essential python Modules to automate various tasks along with their examples for better understanding.

1. Selenium : A powerful tool for automating web browsers, which allows you to interact with web elements programmatically. It's widely used for web scraping, testing and fill forms in web applications.

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.common.keys import Keys
import time

driver = webdriver.Chrome()
driver.get("https://www.google.com")
search_input = driver.find_element(By.NAME, "q")

search_input.send_keys("Python automation")
search_input.send_keys(Keys.RETURN)

time.sleep(60)
driver.quit()

2. PyAutoGUI : This helps us to automate GUI interactions. It is useful for automating tasks involving graphical user interfaces, such as opening applications

import pyautogui
import time
try:
    pyautogui.press('win')
    pyautogui.write('notepad')
    pyautogui.press('enter')
    time.sleep(4)  # Wait for Notepad to open

    pyautogui.write('Hello')
    time.sleep(2)  # Let's type slowly for demonstration
    pyautogui.hotkey('ctrl', 's')
    time.sleep(4)  # Wait for Save As dialog to open
    pyautogui.write('Hello.txt')
    time.sleep(2)  
    pyautogui.press('enter')
    time.sleep(2)  

    print("Task completed successfully!")
except Exception as e:
    print("An error occurred:", e)

3. Requests : This is a simple yet elegant HTTP library for Python, which allows you to send HTTP requests easily. It's commonly used for web scraping, accessing APIs.

import requests
try:
  response = requests.get("https://www.google.com")
  if response.status_code == 200:
      print("Request to Google successful!")
      print("HTML Content:")
      print(response.text)
  else:
      print("Failed to retrieve data from Google. Status code:", response.status_code)
except Exception as e:
  print("An error occurred:", e)

4. Beautiful Soup : It provides us with different functions for parsing HTML and XML documents. It allows you to extract data from web pages effortlessly, facilitating automation of data extraction processes.

from bs4 import BeautifulSoup
import requests

response = requests.get("https://www.example.com")
#Parse the HTML content
soup = BeautifulSoup(response.content, "html.parser")
for link in soup.find_all("a"):
    print(link.get_text())

5. Pandas : pandas is a powerful data manipulation library that help us to form data frames. It's commonly used for automating data processing tasks, such as cleaning, transforming, and analyzing datasets.

 import pandas as pd
 data = {
    'Name': ['Alice', 'Bob', 'Charlie', 'David'],
    'Age': [25, 30, 35, 40],
    'City': ['New York', 'Los Angeles', 'Chicago', 'Houston']
  }
df = pd.DataFrame(data)
print("DataFrame:")
print(df)

Getting Started with Automation

How to Automate a Task: A Step-by-Step Guide

Step 1: Identify Task

Before moving to any planning the first step is to identify the repetitive and time-consuming tasks.

Step 2: Divide the Task into Smaller Steps

Break down the task into smaller, manageable modules. Then start working on each module independently.

Step 3: Research Python Libraries

Research for those Python libraries that can help us in automating each step of the task.

Step 4: Write the Code

Write Python code to automate each step of the task. Given below is a short example to get you started with scripting:

import os
import shutil

file_types = set()
for filename in os.listdir('directory_path'):
    if os.path.isfile(filename):
        file_types.add(filename.split('.')[-1])
        
for file_type in file_types:
    os.makedirs(file_type, exist_ok=True)
    
for filename in os.listdir('directory_path'):
    if os.path.isfile(filename):
        file_type = filename.split('.')[-1]
        shutil.move(filename, os.path.join(file_type, filename))

Step 5: Test the Code

Test the code with sample data to ensure that it performs as expected. Verify that files are organized correctly into their respective folders.

Step 6: Update the Code

Refine and update the code as needed based on test results and feedback. Ensure it handles edge cases and exceptions gracefully.

Introduction to Automation Scripting Languages

Scripting and automation have become indispensable tools for individuals and organizations alike.

Understanding Automation Scripting Languages:
Automation scripting languages are programming languages that are specifically designed to automate repetitive tasks. These languages are having the predefined syntax and functionalities that are useful for automating tasks.

Characteristics of Automation Scripting Languages:

Readable Syntax: Clear and concise code for easier comprehension.
Abundance of Libraries: Extensive pre-built functions streamline development.
Interpretation or Compilation: Code execution either directly (interpreted) or translated into machine language (compiled).
Cross-Platform Compatibility: Capable of running on various operating systems.

Popular Automation Scripting Languages:

Python: Versatile, with rich libraries; used for automation, web dev, and data analysis.
Bash (Shell): Default on Unix; excels in system tasks and command-line ops.
PowerShell: Microsoft's automation tool for Windows; robust system management.

Essential Python Libraries for Automation

OS Module: The os module in Python provides a platform-independent way of interacting with the operating system.

Example :

import os
cwd = os.getcwd()
print("Current directory:", cwd)

new_dir = os.path.join(cwd, 'new_directory')
if not os.path.exists(new_dir):
    os.makedirs(new_dir)
    print("New directory created:", new_dir)
else:
    print("Directory already exists:", new_dir)

files = os.listdir(cwd)
print("Files in current directory:", files)

old_file = os.path.join(cwd, 'old_file.txt')
new_file = os.path.join(cwd, 'new_file.txt')
os.rename(old_file, new_file)
print("File renamed from 'old_file.txt' to 'new_file.txt'")

Note: While renaming the files make sure to create the file whose name you want to change.

Shutil Module: The shutil module offers a high-level interface for file operations, including file copying, moving, and deletion. You can gain more knowledge from shutil module

Example :

import shutil
source_file = 'C:\\Users\\dell\\Desktop\\temp\\source.txt'
destination_file = 'C:\\Users\\dell\\Desktop\\temp\\destination.txt'

try:
    shutil.copy(source_file, destination_file)
    print("File copied successfully!")
except FileNotFoundError:
    print("Source file not found.")
except PermissionError:
    print("Permission denied.")
except Exception as e:
    print("An error occurred:", e)

Note: Make sure to pass the correct path of the source and destination file in the same way as mentioned above.

Subprocess Module: The subprocess module provides a powerful way to spawn new processes. It allows you to execute system commands, run external programs.For more details visit subpress module

Example :

import subprocess
try:
    #Run the command and capture output
    result = subprocess.run(['-l'], capture_output=True, text=True)
    # Print the command output
    print("Command output:", result.stdout)
except FileNotFoundError:
    print("Command not found.")
except Exception as e:
    print("An error occurred:", e)

Time and Datetime Modules: The time and datetime modules in Python provide functionalities for handling time-related tasks and scheduling automation jobs.For further information check Time module in python

Example :

import time
delay = 5
start_time = time.time()
while time.time() < start_time + delay:
    pass
print("Automation job executed after {} seconds.".format(delay))

Note: When this code is executed than after the delay of 5 seconds the message is displayed

Web Scraping and Automation

Web scraping and Automation, empower users to automate repetitive tasks and extract valuable data from websites automatically. The users are able to interact with the web pages and extract data automatically.

Automating Web Tasks
Automation scripts can interact with web pages, submit forms, extract data, and perform other tasks, enabling the automation of web-based workflows.

Example

import requests
from bs4 import BeautifulSoup
#Define the URL of the webpage
url = "https://www.example.com"
try:
  response = requests.get(url)
  if response.status_code == 200:
      print("Webpage accessed successfully!")
      soup = BeautifulSoup(response.text, 'html.parser')
      links = soup.find_all('a')
      for link in links:
          print(link.text)
      # For example, submitting a login form
      payload = {'username': 'your_username', 'password': 'your_password'}
      login_response = requests.post('https://www.example.com/login', data=payload)
      if login_response.status_code == 200:
          print("Login successful!")
      else:
          print("Login failed.")
  else:
      print("Failed to access the webpage.")
except Exception as e:
  print("An error occurred:", e)

Automating System Tasks

Automating system tasks involves streamlining repetitive processes on both Windows and Linux operating systems, enhancing productivity and efficiency.

Task Automation on Windows and Linux

On Windows, PowerShell is a powerful automation tool that provides a wide range of cmdlets and scripts for managing system tasks. With PowerShell, you can automate tasks such as file manipulation, registry editing, user management, and system configuration.
On Linux, shell scripting with Bash is commonly used for automating system tasks. Bash scripts can perform tasks such as directory traversal, process management, and system monitoring.

Example :

$sourceDir = "C:\Users\dell\Desktop\File1.txt"
$destDir = "C:\Users\dell\Desktop\File2.txt"

$timestamp = Get-Date -Format "yyyyMMdd_HHmmss"
$backupFile = "backup_$timestamp.tar.gz"

Compress-Archive -Path $sourceDir -DestinationPath "$destDir\$backupFile"
Write-Host "Backup created: $destDir\$backupFile"

Data Processing and Automation

Introduction to Data Processing

Data processing is the method of transforming raw data into valuable information or insights. It involves several stages such as data collection, cleaning, transformation, analysis, and visualization.

Using Pandas for Data Automation

When the Python library Pandas is used for data automation then it allows efficient processing, manipulation, and analysis of data. Pandas is a powerful library that provides data structures such as DataFrame.

Example :

import pandas as pd
data = pd.read_csv('data.csv')
data.head(7)
df = pd.DataFrame(data)
df.head()
missing_values = df.isna().sum()
print(missing_values)
df_filled = df.fillna(0)
df_filled.head(7)

Testing and Automation

Testing and automation are crucial aspects of software development, ensuring software reliability and quality. It involves systematically evaluating software to find defects and built test cases for testing purpose.

Overview of Testing Automation

Testing automation involves the use of software tools and frameworks to automate the process of testing software applications. One popular testing automation framework is pytest.

Using pytest for Testing Automation

Pytest is a popular testing framework in Python that is used for testing automation. It offers powerful features for writing and executing test cases efficiently. With pytest, developers can easily create test functions, organize test suites, and assert expected outcomes. Its suitable for both small and large-scale projects.

Example :

class Calculator:
    def add(self, a, b):
        return a + b
   def subtract(self, a, b):
         return a - b
calculator = Calculator()
def test_addition():
    assert calculator.add(2, 3)==5
    assert calculator.add(5, -1)==4
def test_subtraction():
    assert calculator.subtract(5, 3)==2
    assert calculator.subtract(10, 7)==3

Note: Always save your `pytest` file with the following naming convention that is `test_filename`.

Real-World Automation Examples

Backup Automation: Schedule regular backups of critical data to prevent loss in case of hardware failure or data corruption.
Server Maintenance: Automate routine server maintenance tasks such as software updates, disk cleanup, and system reboots.
Email Automation: Implement automated email responses, email filtering, and email forwarding to streamline communication processes.
Data Migration: Automate the migration of data between databases or cloud storage solutions to ensure accuracy and efficiency.
Report Generation: Automatically generate and distribute reports on a scheduled basis, reducing manual effort and ensuring timely delivery.
Customer Support: Implement chatbots or automated ticketing systems to handle common customer inquiries and support requests.

Conclusion

In conclusion it is better to say that, Automation with Python empowers individuals and organizations to streamline workflows, increase productivity, and drive innovation. Python's simplicity, versatility, and rich ecosystem of libraries, automation can be used to automate tasks across diverse domains, from file operations and web scraping to system administration and data processing. Do gather some more information by doing some experiment with different libraries and tools, and apply best practices that you learned in this guide to give efficient and reliable automation solutions.

Leveraging Python for System Administration

Wed, 01 May 2024 21:40:14 +0000

In the world of computer systems, keeping things running smoothly is a top priority, and that's where system administrators come in. Python, with its user-friendly interface and powerful capabilities, is like a trusty option. This article dives into how Python empowers system administrators to automate tasks and optimize resources, offering practical insights and real-world examples to support professionals at all levels in mastering this essential tool.

Understanding System Administration

In system administration, you're responsible for managing, maintaining, and operating computer systems, networks, and associated infrastructure within your organization. System administrators, commonly known as sysadmins, are integral in guaranteeing the efficient operation and security of IT systems to uphold business functions.

User Management: This involves managing user accounts, permissions, access rights, password resets, account lockouts, and user group management for effective access control.
File System Management: You'll manage file systems, including tasks like creating, modifying, and deleting files and directories, setting permissions, managing disk space, and monitoring file system health to prevent data loss or corruption.
Software Installation: Installing and configuring software applications involves selecting, installing, and configuring software packages on servers and workstations to ensure optimal performance and security.
Configuration Management: You'll manage the configuration of system components like servers, network devices, and software applications, ensuring consistency, currency, and compliance with organizational standards and best practices.

Challenges in Traditional System Administration

Traditional system administration tasks are known for being manual and repetitive, often requiring sysadmins to perform the same actions across multiple systems. While essential for IT infrastructure functionality and security, these tasks can be time-consuming, error-prone, and resource-intensive.

Volume of Work: Sysadmins handle daily routine tasks such as managing user accounts, software updates, and configuration changes, which can become overwhelming in large-scale environments.
Human Error: Manual tasks can lead to mistakes like misconfigurations, typos, and oversights, resulting in system downtime, data loss, security vulnerabilities, and compliance violations.
Lack of Consistency: Without predefined processes and automation tools, inconsistencies may arise due to ad-hoc methods and shortcuts, leading to compliance issues.
Scalability and Agility: Manual processes hinder scalability and adaptability to changing business needs, becoming increasingly burdensome and inefficient as organizations grow and IT environments become more complex.

In the world of IT, managing large-scale systems is like navigating a maze filled with challenges and complexities. As organizations grow, so does the size and complexity of their systems, presenting sysadmins with a host of hurdles to overcome.

Automation Complexity: Automating large-scale environments is complex and time-consuming, involving the development, testing, and maintenance of automated workflows across diverse systems. This requires careful planning and coordination.
Visibility and Monitoring: Maintaining visibility into large-scale system health, performance, and security is challenging. Insufficient monitoring tools may create blind spots, hindering prompt issue identification and resolution.
Security Risks: Large-scale systems are prime targets for cyber threats. Ensuring robust security measures like access controls, encryption, and threat detection is crucial but challenging in dynamic environments.
Change Management: Implementing changes, updates, and upgrades in large-scale systems requires meticulous planning and coordination.
Compliance and Governance: Comprehensive governance strategies are crucial for documentation, policy enforcement, and compliance across diverse environments.

Essential Python Libraries for System Administration

Python is a valuable asset for system administrators, offering a range of libraries to streamline tasks and boost productivity. These libraries empower sysadmins to interact with operating systems, automate tasks, and manage infrastructure effectively.

os Module: The os module in Python offers a plethora of functions for interacting with the operating system. It enables sysadmins to navigate file systems, manipulate files and directories, and manage processes efficiently.
subprocess Module: The subprocess module in Python allows scripts to spawn new processes, interact with system programs, and execute shell commands. It provides a higher-level interface than the os.system() function.
sys Module: The sys module in Python provides access to system-specific parameters and functions, allowing interaction with the Python interpreter and retrieval of information about the runtime environment, command-line arguments, and standard input/output streams.
shutil Module: The shutil module in Python provides a high-level interface for file and directory operations, including copying, moving, renaming, deleting, archiving, and compressing files.
paramiko Module: The paramiko module in Python is a powerful library for SSH communication and remote server management. It allows sysadmins to securely connect to remote servers, execute commands, transfer files, and manage SSH keys programmatically.
psutil Module: The psutil module in Python offers a cross-platform interface for accessing information on running processes, system utilization, and resources. It enables sysadmins to monitor system performance, manage processes, and collect detailed data on CPU, memory, disk, and network usage.

Note: Although the aforementioned modules usually come pre-installed with Python, you may install a module using the command: `pip install module_name` if it's not present in your Python environment. Import a module using the command: `import module_name`.

Examples of Python Scripts for System Administration

User Management We can create, modify, and delete user accounts using Python by calling system commands (useradd, passwd, userdel) via the subprocess module.

Creating User Account: The process of adding a new user to the system with specified attributes such as username, password, and home directory.
```
  import subprocess
  
  def create_user(username, password):
      try:
          subprocess.run(['powershell', '-Command', f'New-LocalUser -Name {username} -Password (ConvertTo-SecureString "{password}" -AsPlainText -Force)'])
          print(f"User '{username}' created successfully.")
      except subprocess.CalledProcessError as e:
          print(f"Error creating user '{username}': {e}")

  create_user("Sia", "si@123")
```
The equivalent Linux command can be obtained by replacing the main_com line by:
```
  subprocess.run(['useradd', '-m', username])
```
The script uses the subprocess module to execute a PowerShell command, creating a new local user account on a Windows system. The create_user function constructs a PowerShell command with the provided username and password, utilizing the New-LocalUser cmdlet. Upon successful creation, a confirmation message is displayed; otherwise, an error message is shown.

Output:
Modifying User Account: Making changes to an existing user account, such as updating user information or modifying permissions.
```
  import subprocess

  def modify_user(username, new_password):
      try:
          subprocess.run(['powershell', '-Command', f'Set-LocalUser -Name {username} -Password (ConvertTo-SecureString "{new_password}" -AsPlainText -Force)'])
          print(f"Password for user '{username}' modified successfully.")
      except subprocess.CalledProcessError as e:
          print(f"Error modifying password for user '{username}': {e}")

  modify_user("Sia", "si@456")
```
The equivalent Linux command can be obtained by replacing the main_com line by:
```
  subprocess.run(['passwd', username], input=new_password.encode())
```
The script employs the subprocess module to execute a PowerShell command, altering the password of an existing local user account. The modify_user function accepts parameters for the username and the new password, constructing a PowerShell command using string formatting or passwd command. If the operation succeeds, a confirmation message is shown.

Output:
Deleting User Account: Removing an existing user account from the system, including all associated files and directories.
```
  import subprocess

  def delete_user(username):
      try:
          subprocess.run(['powershell', '-Command', f'Remove-LocalUser -Name {username} -Force'])
          print(f"User '{username}' deleted successfully.")
      except subprocess.CalledProcessError as e:
          print(f"Error deleting user '{username}': {e}")

  delete_user("Sia")
```
The equivalent Linux command can be obtained by replacing the main_com line by:
```
  subprocess.run(['userdel', '-r', username])
```
The script uses the subprocess module to execute a PowerShell command, deleting an existing local user account. The delete_user function accepts a username parameter indicating the user account to be deleted. It constructs a PowerShell command using string formatting, utilizing the Remove-LocalUser or userdel cmdlet. After deletion, a success message is displayed.

Output:

Caution: Ensure that the script is executed with appropriate privileges to perform these actions.

Setting User Permissions and Managing Access Control To set user permissions and manage access control with a script, you can utilize the subprocess module in Python to execute system commands for manipulating file permissions and access control lists (ACLs).

import subprocess

def set_permissions(file_path, user, permission):
    try:
        subprocess.run(['icacls', file_path, '/grant', f'{user}:{permission}'])
        print(f"Permissions set successfully for user '{user}' on '{file_path}'")
    except subprocess.CalledProcessError as e:
        print(f"Error setting permissions: {e}")

file_path = "myfile.txt"
user = "Sia"
permission = "F"  # Full control
set_permissions(file_path, user, permission)

The equivalent Linux command can be obtained by replacing the main_com line by:

subprocess.run(['chmod', permission, file_path])

The script uses the icacls command to modify file permissions in Windows. The /grant option specifies user and permissions, while the permission parameter allows customization using flags like F, M, RX, R, W, D.

File System Management To automate file and directory operations such as creation, deletion, and modification in Python, you can use the os module for basic file and directory operations, and the shutil module for more advanced operations like copying and moving files.

File Creation, Modification and Deletion

  import os

  def create_file(file_path):
      try:
          with open(file_path, 'w') as f:
              pass  # Creating an empty file
          print(f"File '{file_path}' created successfully.")
      except Exception as e:
         print(f"Error creating file '{file_path}': {e}")

  def delete_file(file_path):
      try:
          os.remove(file_path)
          print(f"File '{file_path}' deleted successfully.")
      except Exception as e:
          print(f"Error deleting file '{file_path}': {e}")

  def modify_file(file_path, content):
      try:
          with open(file_path, 'w') as f:
              f.write(content)
          print(f"File '{file_path}' modified successfully.")
      except Exception as e:
          print(f"Error modifying file '{file_path}': {e}")

  file_path = "sample.txt"
  create_file(file_path)
  modify_file(file_path, "I hope you are enjoying so far!")
  delete_file(file_path)

create_file creates an empty file at the specified path, modify_file modifies the content of an existing file with the provided content and delete_file deletes the file at the specified path.

Output:

Directory Creation, Modification and Deletion

  import os

  def create_directory(directory_path):
      try:
          os.makedirs(directory_path, exist_ok=True)
          print(f"Directory '{directory_path}' created successfully.")
      except Exception as e:
          print(f"Error creating directory '{directory_path}': {e}")

  def delete_directory(directory_path):
      try:
          os.rmdir(directory_path)
          print(f"Directory '{directory_path}' deleted successfully.")
      except Exception as e:
          print(f"Error deleting directory '{directory_path}': {e}")

  def rename_directory(directory_path, new_name):
      try:
          os.rename(directory_path, new_name)
          print(f"Directory '{directory_path}' renamed to '{new_name}' successfully.")
      except Exception as e:
          print(f"Error renaming directory '{directory_path}': {e}")

  directory_path = "sampledir"
  new_name = "samp_dir"
  create_directory(directory_path)
  rename_directory(directory_path, new_name)
  delete_directory(new_name)

create_directory creates a new directory at the specified path, rename_directory renames an existing directory to the specified new name and delete_directory deletes the directory at the specified path.

Output:

After renaming:

Software Installation and Configuration

Automating Software Installation and Updates Automating software installation and updates using package managers like pip or system-specific tools can be achieved through Python scripts.

For example, to install and update the numpy package:

  import subprocess
   
  def install_package(package_name):
      try:
          subprocess.run(['pip', 'install', package_name])
          print(f"Package '{package_name}' installed successfully.")
      except subprocess.CalledProcessError as e:
          print(f"Error installing package '{package_name}': {e}")
   
  def update_package(package_name):
      try:
          subprocess.run(['pip', 'install', '--upgrade', package_name])
          print(f"Package '{package_name}' updated successfully.")
      except subprocess.CalledProcessError as e:
          print(f"Error updating package '{package_name}': {e}")

  package_name = "numpy"
  install_package(package_name)
  update_package(package_name)

The install_package function installs a Python package using the pip install command. The update_package function updates a Python package to the latest version using the pip install --upgrade command.

Configuring Software Settings and Parameters You can use the sys module in Python to configure software settings programmatically on Windows, particularly for command-line arguments handling.

  import sys
  import subprocess

  def configure_with_tool(tool_path, arguments):
      try:
          subprocess.run([tool_path] + arguments, check=True)
          print("Configuration successful.")
      except subprocess.CalledProcessError as e:
          print(f"Error configuring with tool '{tool_path}': {e}")

  if __name__ == "__main__":
      if len(sys.argv) < 3:
          print("Usage: python script.py <tool_path> <arguments>")
          sys.exit(1)

      tool_path = sys.argv[1]
      arguments = sys.argv[2:]
      configure_with_tool(tool_path, arguments)

When you run the script from the command line, you'll provide the tool path as the first argument as follows:

python Articles/software_settings.py "Articles//SimpleCalculator.exe"

System Monitoring and Reporting You can use Python to gather system metrics such as CPU usage, memory usage, and disk space using various libraries and system commands.

import psutil

def get_cpu_usage():
    cpu_usage = psutil.cpu_percent(interval=1)
    return cpu_usage

def get_memory_usage():
    memory = psutil.virtual_memory()
    total_memory = memory.total
    available_memory = memory.available
    used_memory = memory.used
    memory_usage_percent = memory.percent
    return {
        "total_memory": total_memory,
        "available_memory": available_memory,
        "used_memory": used_memory,
        "memory_usage_percent": memory_usage_percent
    }

def get_disk_usage():
    partitions = psutil.disk_partitions()
    disk_usage = {}
    for partition in partitions:
        partition_usage = psutil.disk_usage(partition.mountpoint)
        disk_usage[partition.mountpoint] = {
            "total": partition_usage.total,
            "used": partition_usage.used,
            "free": partition_usage.free,
            "percent": partition_usage.percent
        }
    return disk_usage

cpu_usage = get_cpu_usage()
memory_usage = get_memory_usage()
disk_usage = get_disk_usage()
print("CPU Usage:", cpu_usage, "%")
print("Memory Usage:", memory_usage)
print("Disk Usage:", disk_usage)

The get_cpu_usage function retrieves CPU usage as a percentage, while get_memory_usage gathers memory statistics, and get_disk_usage collects disk stats. These functions utilize psutil methods for data retrieval.

Output:

Conclusion

System Administration faces challenges with manual tasks and the need for efficiency. Python offers a solution with its libraries like os, subprocess, and shutil, making it ideal for automation. Python scripts can handle user management, permissions, access control, file system operations, software installation, and system monitoring. This empowers administrators to enhance productivity and proactively manage computer systems effectively.