Monitoring Apache on Windows with a simple watchdog

 Scripting, Server, Windows  No Responses »
Jul 262023
 
Apache HTTP server logo

It’d be beautiful if this web server here were stable. Sadly, it is not. It appears that the combination of several technologies or software, both very modern and very old tends to make it fall on its face roughly every 2 – 3 weeks. What certainly doesn’t help is ongoing waves of bots trying to scrape the site. This is bothersome even for modern web servers, but the antique that hosts xin.at simply cannot handle this much. It appears that it is mostly during those phases of massively increased load that the server trips. It may crash entirely or – and this is much more common – stay up and running, but in a hung state, where it would produce nothing but timeouts.

The most likely culprit for this is this cute little button here:

PHP-tan

This is what PHP is actually, truly like

Naturally, another reason might be MS Windows. There are a few peculiarities when running an Apache + MySQL + PHP + SSL stack on older versions of Windows, and who knows, maybe it affects stability as well. Not sure.

In any case, I attempted to write a simple watchdog script to be run every 6 minutes. It would check for the presence of at least one “httpd” process and then probe port 80 to see whether it was online. To hide the terminal window I wrote a very simple VisualBasic Script wrapper monitor-httpd.vbs, which is then launched by the Windows task scheduler for a user with administrative privileges. The wrapper looks somewhat like this:

Set WshShell = CreateObject("WScript.Shell")
WshShell.Run chr(34) & "C:\Scripts\monitor-httpd.bat" & Chr(34), 0
Set WshShell = Nothing

This will launch monitor-httpd.bat without popping up any windows. Originally, that monitor-httpd.bat had looked like this, as described, assuming 10.20.15.100 would be the IP address of your web server and “Apache” the name of your Apache web server system service:

@ECHO OFF
REM Probing for presence of process:
FOR /F "tokens=* delims= usebackq" %%I IN (`pslist.exe ^| grep httpd`) DO SET HTTPDSTATUS=%%I
IF NOT DEFINED HTTPDSTATUS (net.exe start "Apache") ELSE (
  SET HTTPDSTATUS=
  REM Probing for open port to detect application lockups:
  PortQry.exe -n 10.20.15.100 -p TCP -e 80 -q
  REM If locked up, kill Apache (first gracefully, then forcefully) and restart it afterwards:
  IF %ERRORLEVEL% NEQ 0 (
    net.exe stop "Apache"
    pskill.exe httpd
    net.exe start "Apache"
  )
)

As you can see, it makes use of Microsoft “pstools” such as pslist.exe and pskill.exe and PortQry.exe on top of base tools the OS comes with. Sadly, those tools are non-free and I may not redistribute them here. They’re still available online, but unfortunately not in versions compatible with very old Windows releases. This means you’ll have to find your own way of obtaining them or a way to replace them with something else. :(

The above version is also quite flawed. It does detect completely dead web server processes, and processes without a listen socket, but nothing else. The most common crash for me however is that the server would still run and have it’s listen sockets open, but without responding to any requests. What I would need to do is check the actual responsiveness of the server. To see if it’s still serving files. For this, I created a small, empty file, that can be downloaded from the web server, e.g. probefile.

Then, let’s upgrade monitor-httpd.bat a bit:

@ECHO OFF
REM :: This script checks for two things:
REM :: 1. The presence of one or more Apache httpd processes, and if yes...
REM :: 2. ...the server actually responding to requests.
REM ::
REM :: It uses Microsoft's pslist, pskill, net, find and MIT-licensed curl.
REM :: curl license: https://curl.se/docs/copyright.htmll
 
SET logFile="C:\Logs\ApacheWatchdog\apachewatchdog.log"
 
REM :: Probing for presence of the web server process:
FOR /F "tokens=* delims= usebackq" %%I IN (`pslist.exe ^| find "httpd"`) DO SET HTTPDSTATUS=%%I
 
REM :: If the process is nowhere to be found (has crashed), restart it. If it
REM :: is running, we also need to check whether the server is actually
REM :: delivering any files or whether it's in a hung state.
IF NOT DEFINED HTTPDSTATUS (
  net.exe start "Apache"
  DATE /T >>"%logFile%"
  ECHO HTTPd found entirely dead, service restarted>>"%logFile%"
  EXIT /B 1
) ELSE (
  SET HTTPDSTATUS=
  REM :: Try to fetch a special file for this test from the server with curl,
  REM :: try for 10 seconds, then fail.
  curl.exe http://www.yourserver.com:80/probefile --connect-timeout 10 --fail --progress-bar --output /dev/null
  REM :: The most relevant curl return values might be:
  REM ::  0 (success)
  REM :: 22 (HTTP return code over 400, e.g. 404)
  REM :: 28 (Connection timeout)
 
  REM :: If locked up, kill Apache (first gracefully, then forcefully) and
  REM :: restart it afterwards:
  IF %ERRORLEVEL% NEQ 0 (
    net.exe stop "Apache"
    pskill.exe httpd
    net.exe start "Apache"
    DATE /T >>"%logFile%"
    ECHO HTTPd found alive, but hung and unresponsive, service restarted>>"%logFile%"
    EXIT /B 1
  )
)
 
REM :: All's fine at this point, exit gracefully
EXIT /B 0

This isn’t really perfect either. PortQry.exe is gone now, but the pstools are still there. [curl] is now used to check if the server really responds in a meaningful way. Also, optimally we may want to react differently when seeing return value 22 (is the storage backend broken?) and 28 (most likely a hung Apache web server). However, in its current form it will just restart the service no matter what error happened.

It might also be cool to add [BLAT] to send eMails whenever a problem is detected, but let’s leave that out for now.

At least I can provide you with curl in a version compatible with Windows 2000 or newer, built with an old version of CygWin:

Running a CalDAV/CardDAV server on Windows 2000 for multi-user calendars, tasks and contacts synchronization across devices

 Server, Windows  No Responses »
Jul 152022
 
DAV logo

0. Introduction

Recently, I found it more and more troublesome that I couldn’t synchronize my personal calendar(s) across all my devices. Originally, I’ve been running a SyncML server for that purpose, as it was conveniently integrated with my mail server suite. Many years ago I used a Nokia E72 “smart” phone *cough* to synchronize against that server. There is even an app for it on Android called [Synthesis], which I am currectly using on modern Android. However, I could not find any software nor software plugins for SyncML for any one of my Desktop operating systems (WinXP x64, RedHat Enterprise Linux 8 & FreeBSD 12/13). Seems SyncML never really made it onto the desktop. And where it did – like in the form of the Funambol plugin for Mozilla Thunderbird – it’s been long abandoned and no longer works with modern software. Heck, it doesn’t even work with TB 52.9 on XP / XP x64 anymore, it’s too old for even that!

At first I tried to bring SyncML and modern CalDAV/CardDAV solutions together by running my own bridge server using [SyncEvolution] on Linux. In essence, I would run my own DAV server and bridge it to my SyncML server in the background. All of it over HTTPS. DAV as a protocol, originally called WebDAV (Web Distributed Authoring & Versioning) is a series of extensions to HTTP/HTTPS that can be used for sharing and collaboratively working on certain data sets. Like version control for software development (cvs+https://, svn+https:// etc., as opposed to doing it over e.g. svn+ssh://). CalDAV and CardDAV are just altered versions used for calendar, tasks and contact information data.

Unfortunately, that bridge thing did not work. While SyncEvolution can do it the other way around, running as a SyncML server and linking it to a CalDAV/CardDAV backend, it can not run e.g. a CalDAV server and bridge it to a backend SyncML one, like it is in my case. :( Actually, the original developer told me it could, but I just didn’t find any way to do so, even with the latest version.

So, I gave up on the whole idea, when user M477 on the XIN IRC chat suggested just running a DAV server on my stoneage Windows 2000 Server machine directly. I had thought that to be an exercise in futility, but he proved me wrong! This can indeed be done, and when combining the Radicale CalDAV/CardDAV server with modern HTTPS using my own OpenSSL + stunnel backports it can satisfy the security requirements of modern client software as well!

1. The server software

Radicale logoAs for software, some pretty old versions are required, at least partially. I’m using Python 2.7.10, the Radicale 1.1.7 DAV server and as mentioned, my own backport of relatively modern OpenSSL 1.1 and stunnel for bridging HTTPS to a localhost HTTP socket (unpack the files below with [7-Zip]):

1a. Python

Setting up Python is straight-forward, so I won’t discuss this here. Just run the installer and optimally install it into a path not containing any whitespaces.

1b. Radicale

1b1. Installation and basic configuration

Unpack it and put it wherever you wish, e.g. X:\servers\radicale\. It’s all written in interpreted Python language, so to execute it interactively for tests, you’d do something akin to this:

CD /D X:\servers\radicale\
"C:\Program Files\Python27\python.exe" .\radicale.py

I do suggest creating a restricted user to run it as though, just to make sure no part of the software unnecessarily runs with higher privileges. If you wish to create such a user including its user profile folder without having to log in with it interactively, you can run the following as an administrative user on a cmd terminal after regular user creation via Administrative Tools in the system control panel:

runas.exe /profile /user:<username> cmd.exe

This’ll create the profile and give you a terminal where you can work as the target user to set things up. If you’re authenticating against a domain controller’s ActiveDirectory, do it like this:

runas.exe /profile /user:<domainname>\<username> cmd.exe

Note there is a file called config coming with Radicale. This will be expected at the UNIX-style path ~/.config/radicale/config. Python translates the ~ home directory shorthand to %USERPROFILE% on Windows transparently, so pre-create the folders there and copy the config file to that location for the user which will be expected to run the software: %USERPROFILE%\.config\radicale\config. Slash-to-backslash translation will also be done by Python internally, so no need to change the conventions in config.

Edit the config file with your favorite text editor, and take a look at the options hosts, daemon, ssl & realm in the [server] block for now. I’d suggest the following settings:

[server]
hosts = 127.0.0.1:5232
daemon = False
ssl = False
realm = CalDAV/CardDAV

Since we’re not going to rely on the old SSL implementation of Python, the server should listen only on localhost with no encrpytion instead of on the actual, public network. Also, it cannot run as a “daemon”, which is a type of background service on UNIX and Linux, specifically. As for the realm option, you can just enter any arbitrary string. Its value will be presented to you likely as a dialog window title when the server asks your calendar/tasks/contacts client for login information.

1b2. Authentication

Radicale v1 supports multiple authentication backends by default. Newer versions 2 and 3 need additional plugins to gain the same flexibility, but with this version, a lot of them are bundled, like LDAP, IMAP and the file-based htpasswd. Look for the authentication block [auth].

htpasswd-style authentication is probably the easiest. This means creating a users & passwords file using the command line program htpasswd.exe, while picking a compatible hash function that works with Python 2.7.10 (e.g. SHA1). For help on usage, just run htpasswd.exe --help on a cmd terminal with htpasswd.exe on your search path. Creating a new user+password file with SHA1 password hashes is as easy as this:

htpasswd.exe -c -s <passwdfile> <username>

In config, you’d then specify it like this:

[auth]
type = htpasswd
htpasswd_filename = ~/.config/radicale/htpasswd

In my own case, I am running an eMail server with IMAPv4 on the same host, so I chose the IMAP authentication backend instead. As it’s running locally, no SSL is required when authenticating against it:

[auth]
type = IMAP
imap_hostname = localhost
imap_port = 143
imap_ssl = False

1b3. Rights management

Next, look at the [rights] block. For simple setups, where each user only needs to access their own calendar and contacts, I’d suggest setting the rights management type as such: type = owner_only. If you require more complex setups with certain users sharing specific calendars & contacts, I suggest reading up on the from_file value and its implementation. It’s not exactly trivial, but here’s the [documentation]!

1b4. Data storage

The most well-supported way of storing the data on the server side is to do so as such text files. Continue to the [storage] block in config. Let me suggest the following directives:

[storage]
type = filesystem
filesystem_folder = ~/.config/radicale/db

Note that you need to create the folder %USERPROFILE%\.config\radicale\db\ in advance! Also make sure the user who is supposed to be running Radicale has write access to it.

1c. stunnel

Full stunnel documentation with examples towards the end can be found [here]. Documentation on my own Windows 2000 backport can be found [here]. For now, let’s just assume you won’t be changing Radicale’s TCP socket, so it’ll run on localhost:5232. Then, a proper service definition in config\stunnel.conf would be like the following, assuming your public server IP address were 233.204.248.135:

[cdav]
accept  = 233.204.248.135:5232
connect = 5232
cert    = Your-SSL-certificate.pem
key     = Your-SSL-certificate-private-key.pem

Naturally, you’d need to create SSL certificates first. For this I suggest [Let’s Encrypt] if you don’t have any server-side SSL implemented yet. Their certificates are trusted by all modern software vendors, so it’s a good pick when working with modern client software.

1d. instsrv & srvany

Those two are for installing and running interactive programs such as Python as background system services. This is optional, but I’d recommend doing so. Service installation works as follows on a cmd terminal, assuming instsrv.exe is on your search path and srvany.exe is in %WINDIR%\system32\:

instsrv.exe "<service name>" %WINDIR%\system32\srvany.exe

So, e.g.:

instsrv.exe "Radicale CalDAV and CardDAV server" %WINDIR%\system32\srvany.exe

Required system service properties are then edited in the system registry using regedit.exe. Please be careful with that! Look for a registry key named after the service name you just picked, e.g.:

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Radicale CalDAV and CardDAV server\

There need to be three sub-keys: Enum, Parameters and Security. If Parameters doesn’t exist yet, create it, then enter it. Now, here you’ll create three string values called AppDirectory, Application and AppParameters. The first is the current working directory of the server. You’ll enter the path to Radicale as its value. The second is the full path to the Program being called. Enter the full path to python.exe here. And the last is a list of parameters to be passed to the invoked program. In our case this is the full path to the Radicale main program radicale.py. So, for example:

Property Value
Windows Registry string icon AppDirectory X:\servers\radicale
Windows Registry string icon Application C:\Programs\Python27\python.exe
Windows Registry string icon AppParameters X:\servers\radicale\radicale.py

 

Also, don’t forget to go to Win+r, run services.msc and reconfigure that service by setting the user to run the service as, if you’re using a dedicated account for it!

Before starting the service, you should run it interactively on a cmd terminal (e.g. the one you launched in 1b1. with runas.exe). To do that, just replicate the Registry settings on a terminal:

CD /D "X:\servers\radicale\"
"C:\Programs\Python27\python.exe" "X:\servers\radicale\radicale.py"

Make sure it works and you can reach it with a browser. Following the examples above, you’d just need to connect to https://233.204.248.135:5232, and you should get a login prompt. Test authentication, and if it breaks, take a look at the output on the terminal to debug it! The result should be either a blank, white page or alternatively a “Radicale works” message. Once you’re certain it works so far, terminate the program and start the configured system service in its stead, then test again.

2. The client software

In my case, I decided to use three clients: First, [Thunderbird] (both old and new) on Microsoft Windows, Linux and FreeBSD. Second, [KOrganizer] on Linux. And finally, [DAVxâĩ] on Google Android.

When configuring a client for a server-side calendar, the full URL would be in the format https://<server socket>/<user name>/<collection>. A “collection” is essentially a contacts list or a calendar+tasks list. You can pick its name freely when initially creating the collection. E.g. you can just call it “calendar” as well. The user name should match your login credentials.

For example: https://233.204.248.135:5232/johndoe/calendar. Let’s say you own mydomain.org, which points to 233.204.248.135, then of course the following would be much prettier, and won’t trigger any SSL warnings or errors, as long as your SSL certificate’s common name is mydomain.org:

  • https://mydomain.org:5232/johndoe/calendar

In Thunderbird, just switch to the calendars tab and press the “+” symbol on the left pane to add a new one. Select “On the network”, then enter your user name into the “User name” as well as the URL into the “Location” fields. You may also check “Offline Support” to make sure you can see the calendar and get notifications even while your Radicale server is down. Continue and you’ll be promped for a password. Authenticate, then scan for calendars, select the server’s offer and you’re done!

Once the calendar has been created on the server side, you can also import any calendar backups you may have (e.g. from Google Calendar or other software) onto the server. Thunderbird supports the very common iCalendar format (.ics) for this. Wait for the entries to be uploaded, and you should be set!

Here are some example screenshots of two versions of Thunderbird as well as KOrganizer working with a Radicale Server running on ancient Windows 2000, connected via TLSv1.2. Note that there is an additional, local-only calendar added to the mix on Linux (click to enlarge pictures):

 

As for Android, I decided to use the DAVxâĩ app, as mentioned. Setup is pretty similar to the desktop programs, here are some sample screenshots:

 

Note that Apple iOS supports this natively! CardDAV for contact synchronization should be found under Settings / Contacts / Accounts / Add Account / Other, “Add CardDAV Account” and CalDAV for calendar synchronization under Settings / Calendar / Accounts / Add Account, “Other”, “Add a calendar account”, “Add CalDAV Account”. I have not tried this though, as I do not have an iPhone or iPad.

So while I can’t really use good old SyncML in a cross-platform context, this still enables me to share a centralized set of self-hosted contacts, tasks and calendars for me or even for multiple users on an ancient server. While security may be quite debatable in this case, at least the cryptographic part is by no means ancient for both login and data transfer.

While the SyncML server will remain online, XIN eMail server users may now additionally request CalDAV/CardDAV access for synchronizing their calendars, tasks and contacts to an ancient museum server, without having to rely on “the cloud” to hold that data. ;)

Step by step SSL/TLS certificate change around here

 Security, Server  No Responses »
Jun 132022
 
Let's encrypt logo 136

In May 2021, I had [mentioned] that XIN.at had been starting to use Let’s Encrypt certificates with it’s root and intermediate certificates cross-signed by the expired IdenTrust DST Root CA X3 certificate authority. This was done mostly for compatibility reasons, as despite its expiry, some platforms could still trust DST Root CA X3, but not the newer ISRG Root X1. Let’s Encrypt themselves mention Windows XP and Android 7 as such platforms on their [Chain of Trust page]. Recently, some programs, e.g. FileZilla on RedHat Enterprise Linux and probably other systems or R2Mail2 on Android have stopped trusting this cross-signing construct and require constant user intervention to remain operable, which is quite cumbersome for those end users with modern platforms and software.

Let's Encrypt chain of trust since August 2021

Let’s Encrypt chain of trust since August 2021; The DST Root CA X3 is optional (click to enlarge)

On top of that I believe that most Windows XP clients still accessing any services on this server will quite likely be using software integrating their own, much newer cryptographic libraries such as OpenSSL, and their own certificate stores. Examples would include Java, all forks of Mozilla web browsers with XP compatibility and current security fixes, the Thunderbird eMail client, Python and many others.

Hence, I have decided to slowly phase out the DST Root CA X3 cross-signed ISRG Root X1 as well as the equally cross-signed R3 intermediate certificates. From now on, services will be adapted to using this chain:

ISRG Root X1 ―signs→ R3 Intermediate ―signs→ XIN.at subscriber/server certificate

…as opposed to this one:

DST Root CA X3 (expired)―signs―┐
            │                  ↓
          signs         R3 Intermediate ―signs→ XIN.at subscriber/server certificate
            ↓                  ↑
       ISRG Root X1―――signs――――┘

For now, only the FTP+TLS service has been altered as such. The next ones will be the eMail server, the XIN.at webchat frontend, and the main web server, in no particular order.

By now there should be virtually zero clients running into problems as a result of this cleanup!

Also: Please note that while TLS v1.2 and modern cipher suites have been backported for most services on my ancient server, this was so far not possible for the main web server, which will remain using TLS v1.0 and older ciphers. If you cannot access this weblog because of that limitation (e.g. you’re using a modern Google Chrome instead of a reconfigured FireFox), you’ll have to revert to plain, unencrypted HTTP. For most of what few readers stumble over this page that should be fine anyway, given the nature of this site.

Edit 2022-06-22: The certificate chain of the secure IRC server listening at [ircs://xin.at:6697] has now been updated. This does not yet affect the webchat interface at [https://xin.at:8080], which will come next. For very old clients that do not know the ISRG Root X1 CA, you may still be able to connect if your client allows you to enable untrusted certificates, e.g. as X-Chat / HexChat do. Legacy cryptographic protocols implemented for backwards compatibility (e.g. with Windows 9x, 2000, XP, or old Linux 2.x-based systems) will remain in place, namely SSLv3 and TLS v1.0. Aside from having to allow untrusted certificates on very old machines, this should not have any negative effect for users, at least not a prohibitive one.

Edit 2022-06-23: Today, the following services have been updated in the same way: Web mail, web mail administration, SMTPS, IMAPS, POP3S and the [web chat interface]. While doing so, an issue was discovered for the web chat interface while testing it with [testssl]; The server had only provided the subscriber certificate, but no full chain including intermediate and root CA certificates. This was simply a configuration error that has now been corrected!

A (very likely insignificant) SSL certificate change around here

 Server  No Responses »
May 202021
 
Let's encrypt logo 136

Today I’ve been informed by [Let’s Encrypt] that their cross-signing IdenTrust DST root CA X3 certificate will expire on 2021-09-30. So that’s an older authority certificate that your software or operating system has to know to be able to verify that the actual intermediate and client certificates are valid. This one has been used up until now to enable compatibility with very old systems which do not know the newer Let’s Encrypt root certificate, ISRG root X1. This is mostly significant for older network appliances, IoT devices and so on, which simply don’t receive any certificate updates anymore.

For the PC this might affect old operating systems, too. But usually, the operator can just install the ISRG root X1 certificate by themselves on such a machine. I mean, it’s a PC, not a blackbox. Here’s what the current trust chain looks like:

Let's Encrypt chain of trust

Let’s Encrypt chain of trust ([source], click to enlarge)

As you can see, e.g. the intermediates “R3” and “R4” are signed by both the newer ISRG root X1, but also cross-signed with the older DST root CA X3, and in addition to that, DST root CA X3 also signs and trusts ISRG root X1. According to Let’s Encrypt this is also useful for some older Android smartphones.

So, if you get a certificate warning (on top of a TLS v1.0 protocol warning for my server *cough*, can’t upgrade to anything better) when accessing this site after 2021-09-30, you know why.

For most users, nothing will change at all however.

When the time comes, this certificate update will automatically be applied to the following of my services: HTTPS (multiple instances), FTP+S, IRC+S, as well as all encrypted eMail protocols: SMTPS, POP3S & IMAP4S.

Improving commenting on this weblog

 Programming, Server, t3h intarweb  No Responses »
Sep 082020
 
Comment moderation notice logo

On this (still super-slow) weblog, people often sent their comments multiple times, because they thought the first one didn’t go through. I had always assumed this was just performance-related, because people run into white pages of death or timeout errors. But it appears that that alone is not what was responsible for some users’ behavior. It was that the commenting system simply left them in a confused state, which I only noticed today! After a certain WordPress update implementing the original changeset [43436], operators had the option to active a cookie opt-in checkbox below the comment form. This was supposed to make the WordPress commenting system GDPR-compliant, giving the user the choice as to whether they wish to accept cookies. My site isn’t compliant anyway, due to me making use of the Akismet anti-spam system, but yeah [1]. In any case, this update had an unexpected and very much undesired side effect!

If the checkbox remains unchecked, either because the user chose not to opt-in to receive cookies from this page, or because they never even could, because the box wasn’t enabled, comments would go through without the user receiving a “Your comment is awaiting moderation.” notice and without the user seeing their own comments! The page would reload after 10-20 seconds, and there was no change whatsoever, just as if the user had pressed F5 / reload.

Essentially it was as if the user had just sent their text into /dev/null (data oblivion). This understandably left people in a confused state, which is really bad. So some of you chose to try and re-send your comments, further increasing server load in the process (which is significant because the server is from the stone age). So my blog confused users and put my server under needless load – both of which are really bad!

As mentioned before, I had not noticed this until today, and hence never activated that checkbox for users to see. I have switched it on just a few hours ago. From now on, if you reach this page as a new user or an existing user who’s deleted their cookies, you will see this new checkbox when commenting:

Cookie opt-in checkbox

Cookie opt-in checkbox

Now if you check it, everything’s fine, cookie’s being set, and you’ll see your moderation notice. However, if you do not opt-in, the default behavior of older WordPress installations is to not display anything at all. So this was better, but still not the way I wanted it for people to be.

Why should users who choose not to accept cookies have a significantly worse user experience? That’s no good. By now, this issue has been fixed upstream (in some WP 5.x release), so I chose to back-port the patches into this weblog. The corresponding changeset is [44659]. On top of that, I decided to back-port [44681] as well, to further enhance the software’s behavior.

No matter whether you choose to accept cookies for recurring commenting or not, you will now always receive proper feedback from this site, so that you’re not left hanging, wondering what state your comment’s in:

Moderation notice without any cookie being set

Moderation notice without any cookies having been set, giving the commenting user clear feedback

This should improve people’s experience with this site if they choose to write a comment, both with and without cookies.

Please note that the cookies set by this site serve only a single purpose: To allow users to bypass the moderation queue and my approvement of their comments. With the cookie set, only your first comment will need manual approval by me. For the rest of the lifetime of the cookie – currently set to 1 year – you can keep commenting directly, with no approval process standing in your way. Once the cookie expires or you choose to delete it, you will have to have another single comment approved by me.

If you choose not to opt-in to use cookies containing potentially personally identifiable information on this site, every single one of your comments will enter the moderation queue and will thus have to be manually approved by me. With this improvement, this is now the only remaining downside of not accepting cookies here.

My apologies go to you people for having confused you over the last 2 or so years! :oops:

[1] Upon further research, I found that the way this site uses Akismet is actually supposed to be GDPR compliant.

TLS v1.3 is approved: How to compile OpenSSL 1.1.1 with TLS v1.3 support and stunnel 5.45b6 for Windows 2000

 Linux, Programming, Server, Windows  7 Responses »
Mar 292018
 
TLS v1.3 logo

1.) Introduction

Now that the Internet Engineering Task Force has approved the new TLS v1.3 cryptographic protocol, we’ll surely see implementation based on the final specification rather soon, given how far work has progressed in OpenSSL and probably other libraries. In the second to last article I have [shown] how to upgrade servers running on Windows 2000 with modern cryptography and Let’s Encrypt certificates. That software was based on OpenSSL 1.0.2 and stunnel 5.44, which both fully support Windows 2000. However, when looking at the [current state] of the stunnel project in its latest 5.45 beta 6 version, it seems there isn’t even a 32-bit Windows version of it anymore.

Since compiling OpenSSL with some old Microsoft compiler is especially tricky, I tried to cross-compile the latest beta version with mingw 4.9.2 on CentOS 6.9 Linux instead, to get my 32-bit version. The same goes for stunnel. When tried that, I ran into several parts of code that simply can’t work on Windows 2000 any longer as-is.

For my ancient server, TLS v1.3 would be helpful though, as it would lower the CPU load due to its more efficient way of doing TLS handshakes. So there is a pretty good reason for trying this for real.

2.) Required software

Naturally, you need a Linux machine with the 32-bit x86 version of mingw installed for this. It might very well be easy to do on BSDs and other UNIX systems as well. You’ll also need the usual essentials for building programs under Linux like GNU autoconf, make, etc. Then, fetch the latest versions of OpenSSL and stunnel. At the time of writing, that’d be OpenSSL 1.1.1-pre3 and stunnel 5.45b6.

Unpack those, and get your favorite text editor ready, as we’ll need to modify parts of the source code before attempting a build.

3.) OpenSSL with TLS v1.3

First we need OpenSSL, as stunnel will be linking against it. Enter its source directory, then open the header file crypto/bio/bio_lcl.h. After the initial include lines, you’ll need to switch off the AI_PASSIVE macro. This results in OpenSSL using it’s own implementation for name resolutions. If you don’t do this, it would call the Winsock API functions getnameinfo(), getaddrinfo(), freeaddrinfo(), as well as use the struct addrinfo, none of which exist on Windows 2000, but only on Windows XP and newer. So add the following as described, just after the first #include block:

/* XIN.at mod: Disable AI_PASSIVE so we don't call freeaddrinfo()
* and getaddrinfo() etc. on Windows 2000 */
# undef AI_PASSIVE

Save the file, then compile (here, for 16 CPU threads in parallel with make -j16) and install with the following options set:

$ ./Configure enable-tls1_3 no-asm no-async no-dso no-engine --cross-compile-prefix=i686-w64-mingw32- --prefix=/opt/openssl-mingw mingw shared
$ make -j16
# make install

Just a little explanation for the ./Configure options, why they need to be there, and what the implications are:

  1. enable-tls1_3:

    Like the name says, this enables TLS v1.3. Currently, as it’s still only a draft and not the final version, TLS v1.3 won’t be enabled by default, hence this option.

  2. no-asm:

    This is optional depending on your hardware. If you remove it, OpenSSL will be built including SSE2 code, so you’d need to run Windows on a machine with a new enough processor, meaning either Intels’ Pentium 4 or AMDs’ Athlon 64 CPU. With that option gone, it won’t work on older chips. As my Pentium Pro CPUs don’t have any SSE, I have to fall back to pure C code without assembly optimizations by setting this parameter.

  3. no-async:

    This disables asynchronous sockets in WinSock. This is required as those need light-weight co-operatively multitasking threads called “Fibres”, which don’t exist on Windows 2000. This disables the otherwise failing calls ConvertFiberToThread() and ConvertThreadToFiber().

  4. no-dso:

    Disables the shared object abstraction layer, and with it the call to GetModuleHandeEx(), which is used to communicate with kernel drivers for cryptographic hardware acceleration engines. This also means that we have to switch off support for things like VIA Padlock unfortunately. So no more hardware acceleration, as no-dso implies no-engine.

  5. no-engine:

    Required by no-dso. This disables all cryptographic hardware acceleration engines. As a side-effect, this also disables the native Windows CryptAPI/schannel support of stunnel, so you can no longer store certificates in Windows’ own certificate store either. For us, the CryptAPI is useless anyway, as it’s far too old on Windows 2000, no matter how you look at it (SSL v3, TLS v1.0, SSL_RSA_WITH_RC4_128_MD5, SSL_RSA_WITH_RC4_128_SHA, SSL_RSA_WITH_3DES_EDE_CBC_SHA).

  6. --cross-compile-prefix:

    Here: A mingw running on a 64-bit host operating system targeting a 32-bit Windows system. You can determine this string by the names of your mingw programs. Usually that string triplet should be correct as-is for what we’re doing.

  7. --prefix:

    Don’t change it! This is the installation directory for our cross-compiled Windows version of OpenSSL, and this location is exactly where the stunnel build system will look for it!

Allright, that settles it with OpenSSL. Next: stunnel!

4.) stunnel â‰Ĩ5.45

Enter stunnels’ source directory, and open src/str.c in a text editor. Then, look for the following code block:

expand/collapse source code
/* reportedly, malloc does not always return 16-byte aligned addresses
* for 64-bit targets as specified by
* https://msdn.microsoft.com/en-us/library/6ewkz86d.aspx */
#ifdef USE_WIN32
#define system_malloc(n) _aligned_malloc((n),16)
#define system_realloc(p,n) _aligned_realloc((p),(n),16)
#define system_free(p) _aligned_free(p)
#else
#define system_malloc(n) malloc(n)
#define system_realloc(p,n) realloc((p),(n))
#define system_free(p) free(p)
#endif

 

This shows that a fix of some memory allocation functions has been implemented to correct the old functions’ behavior for 64-bit code. However, this also replaces the functions for 32-bit versions of Windows with aligning ones. Problem: Those don’t exist on Windows 2000 either. We’re talking about _aligned_malloc(), _aligned_realloc() and _aligned_free() replacing the good old malloc(), realloc() and free() functions.

Mind you, the x86 architecture isn’t as strict as not to allow for unaligned memory access, but the CPU fixing things transparently in the background does reduce performance when misaligning your accesses to RAM (You can compare this to filesystem misalignments on solid state drives). So forcing the use of memory alignment is a good thing. But it’s not something Windows 2000 needs, as for 32-bit, the old functions should align properly anyway. Replace the above code with the following:

expand/collapse source code
/* XIN.at mod: Removed aligning memory functions required for x86_64
* to restore Windows 2000 compatibility. Will be using the regular calls
* instead, as they should work fine for x86_32 */
#define system_malloc(n) malloc(n)
#define system_realloc(p,n) realloc((p),(n))
#define system_free(p) free(p)

 

Now run the following commands for a cross-compiled build of stunnel linked against the modern OpenSSL you’ve built before. In my case, I’m doing a parallel build across 16 CPU threads again. What ./configure does might look wrong to you in terms of cross-compiling, but just ignore that. It’s going to work just fine (Hopefully at least, heh)!

$ ./configure
$ cd src/
$ make -j16 mingw
$ cd ../doc/
$ make
$ cd ..

And we’re set! All you have to do now is copy off the required files to replace your original stunnel installation. The following files are required; On the left side: The location of the file on a typical Windows installation of stunnel. On the right side: Where to find the corresponding new file on your Linux machine. You might need to adjust the paths a little depending on where you have extracted the source code and where exactly mingw is installed. Also, your current installation might not yet have libssp-0.dll, but you’ll need that one too:

  • %PROGRAMFILES%\stunnel\bin\libssp-0.dll <- /usr/i686-w64-mingw32/sys-root/mingw/bin/libssp-0.dll
  • %PROGRAMFILES%\stunnel\bin\openssl.exe <- /opt/openssl-mingw/bin/openssl.exe
  • %PROGRAMFILES%\stunnel\bin\libssl-1_1.dll <- /opt/openssl-mingw/bin/libssl-1_1.dll
  • %PROGRAMFILES%\stunnel\bin\libcrypto-1_1.dll <- /opt/openssl-mingw/bin/libcrypto-1_1.dll
  • %PROGRAMFILES%\stunnel\config\openssl.cnf <- /opt/openssl-mingw/ssl/openssl.cnf
  • %PROGRAMFILES%\stunnel\bin\stunnel.exe <- ~/yourbuilddir/stunnel/bin/mingw/stunnel.exe
  • %PROGRAMFILES%\stunnel\bin\tstunnel.exe <- ~/yourbuilddir/stunnel/bin/mingw/tstunnel.exe
  • %PROGRAMFILES%\stunnel\doc\stunnel.html <- ~/yourbuilddir/stunnel/doc/stunnel.html

5.) Using it

stunnel with the modern OpenSSL can be used as-is. Just copy the files over your existing installation of stunnel 5.44 or older, then restart the program or service.

Other applications which do not work in tandem with stunnel might need to be recompiled though. There is some software where OpenSSL DLLs can just be swapped out all the way from version 0.9.6 to 1.0.2, but for those, it failed with 1.1.1 for me. The DLLs just wouldn’t load even if a full [dependency walk] checks out ok. Could also be because they’ve been built with mingw instead of Microsoft Visual C though, who knows. For closed source software or open source software that is too hard to recompile, using stunnel is usually a good alternative, as has been shown [here].

6.) The truly final TLS v1.3

As soon as OpenSSL 1.1.1 gets released with official TLS v1.3 support, I will update this article in case the process shown here will require any changes to work with the final versions.

Anyway, I’m looking forward to how much speed can be gained using TLS v1.3 over TLS v1.2 on CPUs from 1997! ;)

7.) Hey, I just want the binary programs to use on Windows 2000!

Oh my… Well, here you go:

As for the source code, you can find the original sources of OpenSSL [here] and the stunnel sources [here]. The patches shown by me here shall be released under the [OpenSSL license] for the code added to OpenSSL and under the [GPL v3] for the code added to stunnel.

Two new features on this weblog: HTTPS and support for mobile devices

 Server  No Responses »
Mar 222018
 
Responsive Design with Encryption logo

1.) Encryption on this weblog

[1][2] In the last article you can see how I recently managed to implement Let’s Encrypt SSL certificates on Windows 2000 Server, including regular renewals via an ACME client that works on said platform. Since I believe in user choice though, I decided not to force people to use encryption on my web sites, as it also lowers performance even further on my already extremely slow server. Instead, I aimed to make this blog software and all of its plugins fully protocol-independent, which proved to be quite a piece of work requiring modification of the main and theme source code, massive changes in the database (lots of static HTTP:// links there) and also changes to one of the CSS stylesheets and the code generating it. But with that done, the visiting user may choose whether to use encryption or whether not to. For this, the weblogs subdomain has been added to the Let’s Encrypt multi-domain SSL certificate.

So from now on, you can visit this place either in an unencrypted fashion via [http://wp.xin.at], or in an encrypted one via [https://wp.xin.at]!

Encryption on the desktop

Encryption on the desktop, finally working after several days of work invested

2.) Mobile-friendly design

Now, before that I had worked on another update solely for this weblog, which is to finally make it mobile device friendly. Reading the desktop version of this site is quite painful on smartphones and even tablets after all. Don’t get me wrong, my main target audience will still be people sitting in front of PCs and workstations, but still, I wanted to make this work. Sometimes even I dig up some of my own pages on a smartphone after all.

Instead of using a fully responsive design – which I just can’t bring myself to like at all – I installed a plugin that allows me to have a secondary mobile theme solely for smartphones and tablets. This is a better solution for now I think, as it makes the site very low-traffic and pretty fast on mobile devices without the need to throw away the current desktop theme. This is what it looks like on a Blackberry KEYone Android 7.1.1 smartphone at a 1080×1620 resolution:

 

And thank god, the mobile design plugin didn’t break the encryption like several other plugins did! It just works with both protocols now, so the mobile site can be reached via HTTP:// and HTTPS:// as well:

Encryption on mobile devices

Encryption also works on mobile devices (Click to enlarge)

There are still a few things that are a bit broken for the mobile version though. I hadn’t thought of setting a “featured image” for each article, instead inserting a small logo on the top left of each article manually. I have updated most recent articles to use a featured image, but the majority of posts still hasn’t been updated, so older posts don’t have those distorted little logos in their mobile versions yet.

The bigger problem is the source code though. The mobile version renders the HTML lists used for line numbers very wrongly, so source code with line numbers looks like total crap. Probably not something I can easily fix, but maybe I’ll look into it in the future. It’s likely going to require a change somewhere in the CSS stylesheets (not that I understand CSS all that well).

If there are any issues with either the encrypted version of the site (like if you encounter pages with a broken padlock in the browsers’ address bar due to mixed content) or with the mobile theme, please report them in the comments! I will fix them – if I can.

[1] The lock icon is ÂĐ by Svengraph and is licensed under the CC-BY 3.0 unported license

[2] The responsive design logo is ÂĐ by Encinitas Web Design and is licensed under a not otherwise specified CC license

Fixing and (re)compiling ancient software: The AnGeL IRC bot

 Server, Windows  No Responses »
Oct 092017
 
Visual Basic 6.0 logo

A long, long time ago, there was a pretty useful little IRC bot for Windows called the [AnGeL] bot. It seems like nobody remembers it anymore, but it was born from the Anime / Sailor Moon scene back in the late 90s to early 2000s and developed by a German software engineer going by the name of [Benedikt HÞbschen]. The bot was pretty widespread for a while, at least in the German speaking parts of the Internet, and it was extensible by writing VBScript code using Microsofts’ Windows Script Host.

So, essentially, it was what you’d have used if you couldn’t run UNIX or Linux with an eggdrop bot. And I sure couldn’t, because back then I barely even knew about the existence of such systems.

Recently, I ran into a small little problem though; I wanted the bot to create and maintain an SSL/TLS-only channel. So, an IRC channel that would let users join and chat with each other only if they’re connecting to the IRC server via an encrypted connection. This is usually done by setting the +z flag on the channel, which might be followed by the IRC server itself also setting the encryption indicator mode +Z automatically.

However, I found that the bot wouldn’t enforce +z at all. It wouldn’t even set the mode when asked to do so explicitly. It was possible to add it to the list of enforced modes, but it just wouldn’t work, with the same being true for +P (protect channel modes even when nobody is in the channel).

Luckily, Mr. HÞbschen made the source code available under the GPL license (that’s what he told me personally) [here]! And yes, that is VisualBasic 6 code. And yes, VB6 is a part of the infamous Visual Studio 6, you might know the abbreviation “VC6” from C/C++ programs compiled with it. So I though I’d inspect the source code and attempt to fix this issue.

I fired up my Windows 2000 Pro SP4 virtual machine for that, installed Visual Studio 6 (thank you, MSDNAA/Dreamspark/Imagine) and its service pack 6, and then loaded the project file:

The AnGeL IRC bot source code loaded in VB6

That development environment is ancient, and it sure looks the part… what a mess.

I identified the part of the code that would need changing, it’s in the public function ChangeMode() in SourceCode/Modules/Server/Server_Functions.bas. I simply copied some code and adapted it for my purpose, adding just +z and +P support for now:

Server_Functions.bas, expand/collapse source code
  1. Public Function ChangeMode(Should As String, Current As String) ' : AddStack "Routines_ChangeMode(" & Should & ", " & Current & ")"
  2. Dim u As Long, CurMode As Long, Changes As String, InsertWhat As String
  3. Dim CurPos As Long, LimitPos As Long, KeyPos As Long
  4.   ' Added by GrandAdmiralThrawn (http://wp.xin.at/archives/4343):
  5.   ' modes +z (SSL/TLS enforce) and +P (permanent channel with
  6.   ' modes preservation even when empty):
  7.   CurMode = GetModeChar(Current, "z")
  8.   Select Case GetModeChar(Should, "z")
  9.     Case -1: If CurMode = 1 Then Changes = Changes & "-z"
  10.     Case 1: If CurMode = 0 Then Changes = Changes & "+z"
  11.   End Select
  12.   CurMode = GetModeChar(Current, "P")
  13.   Select Case GetModeChar(Should, "P")
  14.     Case -1: If CurMode = 1 Then Changes = Changes & "-P"
  15.     Case 1: If CurMode = 0 Then Changes = Changes & "+P"
  16.   End Select
  17.   ' End of modification by GAT.
  18.   CurMode = GetModeChar(Current, "p")
  19.   Select Case GetModeChar(Should, "p")
  20.     Case -1: If CurMode = 1 Then Changes = Changes & "-p"
  21.     Case 1: If CurMode = 0 Then Changes = Changes & "+p"
  22.   End Select
  23.   CurMode = GetModeChar(Current, "s")
  24.   Select Case GetModeChar(Should, "s")
  25.     Case -1: If CurMode = 1 Then Changes = Changes & "-s"
  26.     Case 1: If CurMode = 0 Then Changes = Changes & "+s"
  27.   End Select
  28.   CurMode = GetModeChar(Current, "m")
  29.   Select Case GetModeChar(Should, "m")
  30.     Case -1: If CurMode = 1 Then Changes = Changes & "-m"
  31.     Case 1: If CurMode = 0 Then Changes = Changes & "+m"
  32.   End Select
  33.   CurMode = GetModeChar(Current, "t")
  34.   Select Case GetModeChar(Should, "t")
  35.     Case -1: If CurMode = 1 Then Changes = Changes & "-t"
  36.     Case 1: If CurMode = 0 Then Changes = Changes & "+t"
  37.   End Select
  38.   CurMode = GetModeChar(Current, "i")
  39.   Select Case GetModeChar(Should, "i")
  40.     Case -1: If CurMode = 1 Then Changes = Changes & "-i"
  41.     Case 1: If CurMode = 0 Then Changes = Changes & "+i"
  42.   End Select
  43.   CurMode = GetModeChar(Current, "n")
  44.   Select Case GetModeChar(Should, "n")
  45.     Case -1: If CurMode = 1 Then Changes = Changes & "-n"
  46.     Case 1: If CurMode = 0 Then Changes = Changes & "+n"
  47.   End Select
  48.   If InStr(ServerChannelModes, "c") Then
  49.     CurMode = GetModeChar(Current, "c")
  50.     Select Case GetModeChar(Should, "c")
  51.       Case -1: If CurMode = 1 Then Changes = Changes & "-c"
  52.       Case 1: If CurMode = 0 Then Changes = Changes & "+c"
  53.     End Select
  54.   End If
  55.   If InStr(ServerChannelModes, "C") Then
  56.     CurMode = GetModeChar(Current, "C")
  57.     Select Case GetModeChar(Should, "C")
  58.       Case -1: If CurMode = 1 Then Changes = Changes & "-C"
  59.       Case 1: If CurMode = 0 Then Changes = Changes & "+C"
  60.     End Select
  61.   End If
  62.  
  63.   For u = 1 To Len(Should)
  64.     Select Case Mid(Should, u, 1)
  65.       Case "l": If GetModeChar(Should, "l") = 1 Then CurPos = CurPos + 1: LimitPos = CurPos + 1
  66.       Case "k": CurPos = CurPos + 1: KeyPos = CurPos + 1
  67.       Case " ": Exit For
  68.     End Select
  69.   Next u
  70.  
  71.   CurMode = GetModeChar(Current, "l")
  72.   Select Case GetModeChar(Should, "l")
  73.     Case -1: If CurMode = 1 Then Changes = Changes & "-l"
  74.     Case 1
  75.       If CurMode = 0 Then Changes = Changes & "+l": InsertWhat = " " & Param(Should, LimitPos)
  76.       If CurMode = 1 Then If Param(Current, 2) <> Param(Should, LimitPos) Then Changes = Changes & "+l": InsertWhat = " " & Param(Should, LimitPos)
  77.   End Select
  78.   CurMode = GetModeChar(Current, "k")
  79.   Select Case GetModeChar(Should, "k")
  80.     Case -1: If CurMode = 1 Then Changes = Changes & "-k" & InsertWhat & " " & Param(Current, ParamCount(Current)): InsertWhat = ""
  81.     Case 1: If CurMode = 0 Then Changes = Changes & "+k" & InsertWhat & " " & Param(Should, KeyPos): InsertWhat = ""
  82.   End Select
  83.  
  84.   Changes = CleanModes(Changes)
  85.   ChangeMode = Changes & InsertWhat
  86. End Function

Honestly, I didn’t think it would actually compile at all. But just hit File \ Make AnGeL.exe, and it all builds just fine! And it’s fast as well. At least the building process is.

I chose to have VB6 “Optimize for fast code” and to have it “favor Pentium Pro(tm)”, whatever that means. But I assume it’s faster on P6 / i686 architectures now (Pentium Pro, Pentium II/III and other more modern chips). Probably also requires such a processor now, breaking compatibility with 586 and earlier chips, but I’m not sure whether that’s true.

I gave it the version number 1.6.3, with the latest I could ever find on the web before having been 1.6.2. You can download this version together with the source code here:

If you just want to run it instead of an existing one, all you need to do is to copy the AnGeL.exe over the one you have now, and that’s it. To edit the code, you need VisualBasic 6, just load the project file ANGEL.VBP and you can start to modify and recompile it.

Hah, changing the AnGeL bot and building its source code after so many years… felt a little bit like touching the holy grail or something. ;)

My thanks fly out for Benedikt HÞbschen for developing the AnGeL bot, and for open-sourcing it! Also, I would like to thank all the contributors to the project as well! I’ll continue to use the bot, probably for a long time to come. :)

Monitoring & restarting an unstable Windows service automatically

 Scripting, Server, Windows  No Responses »
Nov 242016
 
Broken Windows logo

[1] I know what I should do if a system service on Microsoft Windows starts crashing of course; Fixing it is the way to go! But sometimes you simply can’t, because the component causing a certain instability can’t be swapped out or updated. Now Windows services do have a mechanism for monitoring and restarting a service upon failure, but it seems that only works if the system gets an actual error code back from the service upon termination. But it doesn’t seem to work (at least for me) if the service just dies abnormally. Windows recognizes the service has stopped somehow of course, but the restart procedure just doesn’t kick in.

So I thought I’d do it myself, programmatically. And it’s actually pretty easy. I solved this with VBScript, Windows Batch and Mark Russinovichs’ pslist plus grep. So the prerequisites are:

  • Microsoft Windows (well, huh..)
  • MS Windows Script(ing) Host / VBScript, Windows should come with this preinstalled since Windows 2000.
  • [pslist]
  • [grep][src] (grep is optional, I used GNU grep 2.5.4 in this case, licensed under the [GPLv3+])

Make sure the pstools and grep are within your %PATH%, so Windows can find those .exe files. If you don’t want to use grep, you can also use Microsofts’ own find command, if your version of Windows has it.

I divided this into two small scripts. Since the main part is Batch, it might be problematic if you run it at very short intervals, checking for the services’ status, because you get a command window popping up on the desktop. Since most users wouldn’t want that, another script acts as a launcher, hiding the cmd.exe window so it’s run fully in the background without disturbing any potential users or administrators. The launcher looks like this, in my case it’s meant to watch over an Apache web server:

  1. Set WshShell = CreateObject("WScript.Shell")
  2. WshShell.Run chr(34) & "C:\Server\Scripts\monitor-httpd.bat" & Chr(34), 0
  3. Set WshShell = Nothing

And that script C:\Server\Scripts\monitor-httpd.bat we’re launching looks like this:

  1. @ECHO OFF
  2. FOR /F "tokens=* delims= usebackq" %%I IN (`pslist ^| grep httpd`) DO SET HTTPDSTATUS=%%I
  3. IF NOT DEFINED HTTPDSTATUS (net start "Apache2.2") ELSE (SET HTTPDSTATUS=)

A version relying on Microsoft find instead of GNU grep could look like this:

  1. @ECHO OFF
  2. FOR /F "tokens=* delims= usebackq" %%I IN (`pslist ^| find /I "httpd"`) DO SET HTTPDSTATUS=%%I
  3. IF NOT DEFINED HTTPDSTATUS (net start "Apache2.2") ELSE (SET HTTPDSTATUS=)

To get a services’ exact name, just launch services.msc from Start \ Run or run the command net start on a cmd terminal.

As you can see, this greps “httpd” from the process list and pushes its output into %%I and finally into %HTTPDSTATUS%. We have to use a FOR /F for that, as Windows has no way of pushing command outputs from subshells into shell variables like UNIX has (like e.g. var=`command` or var=$(command)). Then we check for the status of that variable. If it’s not defined, then the process http.exe was nowhere to be found! In that case we restart the associated system service (needs proper permissions!). If the variable is defined, we do nothing but unsetting it, since we can assume the service is operating normally. Or at the very least it’s running. ;)

You can automate that by using the Windows task scheduler:

Scheduling an Apache web server "watchdog"

Scheduling an Apache web server “watchdog” (German Windows)

Create a Schedule to your liking and you’re done! If you can afford the affected service to be down for 5 minutes and no longer, just run it every 4 minutes or so.

The solution shown above can easily be adapted to monitor and restart any Windows service you have, as long as the service isn’t fundamentally broken so that it wouldn’t even start up anymore. Also, you can do a lot more, like sending notification eMails with a command line mailer like [blat] when crashes do occur. Of course, this is only useful for services that crash rarely. If it dies every few minutes, you should reaaally fix it instead of just pushing the restart button all the time… ;)

And that’s that!

[1] ÂĐ Mar.0007. Original Version for desktopwallpapers4.me.

How to run the IBM SCSI ServeRAID manager on FreeBSD UNIX

 Security, Server, Storage, Unix  No Responses »
Nov 222016
 
FreeBSD IBM ServeRAID Manager logo

And yet another FreeBSD-related post: After [updating] the IBM ServeRAID manager on my old Windows 2000 server I wanted to run the management software on any possible client. Given it’s Java stuff, that shouldn’t be too hard, right? Turned out not to be too easy either. Just copying the .jar file over to Linux and UNIX and running it like $ java -jar RaidMan.jar wouldn’t do the trick. Got nothing but some exception I didn’t understand. I wanted to have it work on XP x64 (easy, just use the installer) and Linux (also easy) as well as FreeBSD. But there is no version for FreeBSD?!

The ServeRAID v9.30.21 manager only supports the following operating systems:

  • SCO OpenServer 5 & 6
  • SCO Unixware 7.1.3 & 7.1.4
  • Oracle Solaris 10
  • Novell NetWare 6.5
  • Linux (only certain older distributions)
  • Windows (2000 or newer)

I started by installing the Linux version on my CentOS 6.8 machine. It does come with some platform-specific libraries as well, but those are for running the actual RAID controller management agent for interfacing with the driver on the machine running the ServeRAID controller. But I only needed the user space client program, which is 100% Java stuff. All I needed was the proper invocation to run it! By studying IBMs RaidMan.sh, I came up with a very simple way of launching the manager on FreeBSD by using this script I called serveraid.sh (Java is required naturally):

  1. #!/bin/sh
  2.  
  3. # ServeRAID Manager launcher script for FreeBSD UNIX
  4. # written by GAT. http://www.xin.at/archives/3967
  5. # Requirements: An X11 environment and java/openjdk8-jre
  6.  
  7. curDir="$(pwd)"
  8. baseDir="$(dirname $0)/"
  9.  
  10. mkdir ~/.serveraid 2>/dev/null
  11. cd ~/.serveraid/
  12.  
  13. java -Xms64m -Xmx128m -cp "$baseDir"RaidMan.jar com.ibm.sysmgt.raidmgr.mgtGUI.Launch \
  14. -jar "$baseDir"RaidMan.jar $* < /dev/null >> RaidMan_StartUp.log 2>&1
  15.  
  16. mv ~/RaidAgnt.pps ~/RaidGUI.pps ~/.serveraid/
  17. cd "$curDir"

Now with that you probably still can’t run everything locally (=in a FreeBSD machine with ServeRAID SCSI controller) because of the Linux libraries. I haven’t tried running those components on linuxulator, nor do I care for that. But what I can do is to launch the ServeRAID manager and connect to a remote agent running on Linux or Windows or whatever is supported.

Now since this server/client stuff probably isn’t secure at all (no SSL/TLS I think), I’m running this through an SSH tunnel. However, the Manager refuses to connect to a local port because “localhost” and “127.0.0.1” make it think you want to connect to an actual local RAID controller. It would refuse to add such a host, because an undeleteable “local machine” is always already set up to begin with, and that one won’t work with an SSH tunnel as it’s probably not running over TCP/IP. This can be circumvented easily though!

Open /etc/hosts as root and enter an additional fantasy host name for 127.0.0.1. I did it like that with “xin”:

::1			localhost localhost.my.domain xin
127.0.0.1		localhost localhost.my.domain xin

Now I had a new host “xin” that the ServeRAID manager wouldn’t complain about. Now set up the SSH tunnel to the target machine, I put that part into a script /usr/local/sbin/serveraidtunnel.sh. Here’s an example, 34571 is the ServeRAID agents’ default TCP listen port, 10.20.15.1 shall be the LAN IP of our remote machine hosting the ServeRAID array:

#!/bin/bash
ssh -fN -p22 -L34571:10.20.15.1:34571 mysshuser@www.myserver.com

You’d also need to replace “mysshuser” with your user name on the remote machine, and “www.myserver.com” with the Internet host name of the server via which you can access the ServeRAID machine. Might be the same machine or a port forward to some box within the remote LAN.

Now you can open the ServeRAID manager and connect to the made-up host “xin” (or whichever name you chose), piping traffic to and from the ServeRAID manager through a strongly encrypted SSH tunnel:

IBM ServeRAID Manager on FreeBSD

It even detects the local systems’ operating system “FreeBSD” correctly!

And:

IBM ServeRAID Manager on FreeBSD

Accessing a remote Windows 2000 server with a ServeRAID II controller through an SSH tunnel, coming from FreeBSD 11.0 UNIX

IBM should’ve just given people the RaidMan.jar file with a few launcher scripts to be able to run it on any operating system with a Java runtime environment, whether Windows, or some obscure UNIX flavor or something else entirely, just for the client side. Well, as it stands, it ain’t as straight-forward as it may be on Linux or Windows, but this FreeBSD solution should work similarly on other systems as well, like e.g. Apple MacOS X or HP-UX and others. I tested this with the Sun JRE 1.6.0_32, Oracle JRE 1.8.0_112 and OpenJDK 1.8.0_102 for now, and even though it was originally built for Java 1.4.2, it still works just fine.

Actually, it works even better than with the original JRE bundled with RaidMan.jar, at least on MS Windows (no more GUI glitches).

And for the easy way, here’s the [package]! Unpack it wherever you like, maybe in /usr/local/. On FreeBSD, you need [archivers/p7zip] to unpack it and a preferably modern Java version, like [java/openjdk8-jre], as well as X11 to run the GUI. For easy binary installation: # pkg install p7zip openjdk8-jre. To run the manager, you don’t need any root privileges, you can execute it as a normal user, maybe like this:

$ /usr/local/RaidMan/serveraid.sh

Please note that my script will create your ServeRAID configuration in ~/.serveraid/, so if you want to run it as a different user or on a different machine later on, you should recursively copy that directory to the new user/machine. That’ll retain the local client configuration.

That should do it! :)

 Older Entries