A Threshold Autoregressive Model for Software Aging

Xiu-E Chen, Quan Quan, Yun-Fei Jia and Kai-Yuan Cai

Department of Automatic Control

Beijing University of Aeronautics and Astronautics

Beijing 100083, China

xiuechen@asee.buaa.edu.cn  

xiuechen@gmail.com

Abstract

Long running software systems such as client-server type systems are known to experience an aging phenomenon called software aging, one in which the accumulation of errors during the execution of software leads to performance degradation and eventually results in failure. To study and counteract the phenomenon of software aging, we collect and log data on several system resource usage and activity parameters of a web server. Based on the experimental results, we argue that software aging process could be divided into four stages: robust stage, transition stage, failure-probable stage and failure stage. A non-linear threshold autoregressive (TAR) model is then proposed to model and forecast the resource usage in the respective stages. In comparison with AR model, this modeling method is more effective.

1. Introduction

The phenomenon of “software aging” in which the state of the software degrades with time has been reported in recent literature [1, 2, 3].

Software researches could be classified into theoretical researches, experimental researches, and engineering researches [5]. Present study on software aging could also be divided into two classes: model-based and measurement-based. Measurement-based approaches primarily validate the existence of software aging through measuring various parameters of computer system. Ref. [2] monitors various parameters of a server and then construct ARX model for resource parameters to estimate resource exhaustion time due to software aging. The methods of fractal analysis are applied online to detect the trend of performance degradation [3].

Compared with previous work, the main contribution of our paper is as follows: first, in our experiments, we adopt the experimental platform in [2] and improve it. Three client machines instead of one are used to simulate concurrent requests in real world; second, a non-linear TAR model is proposed to analyze and forecast the resource usage in different system stages. Compared with previously developed linear models, a non-linear model better reveals the heavily irregular and nonlinear nature of monitored resource usage. TAR model is also more efficient to describe resource usage in the respective stages in process of software aging.

2. Experiments

2.1. Experimental setup

The components and the system structure are illustrated in Figure 1.

Figure 1. Experimental setup

Different from [2], the system consists of three clients instead of one to simulate concurrent requests in real world and accelerate the process of software aging.

Within all the parameters monitored by top utility and httperf [4], we only concentrate on six of them:

1.        phymemfree-the amount of free physical memory.

2.        buffers-the amount of memory used for buffers.

3.        cache-the amount of memory used as page cache.

4.        swapused-the amount of used swap space.

5.        replytime-the interval between the time httperf sends out the first byte of request and the time it receives the first byte of reply.

2.2. Experiment I: variable workload          

In Experiment I, we imposed variable workload on the web server. The connection rate generated by httperf varied between 30 requests/sec and 960 requests/sec.

Figure 2. Connection rate in experiment I

(a)

(b)

Figure 3.Test results in Experiment I

From the plots in Figure 3, one can conclude that resource usage does reflect the performance of the server. The coefficients of correlation between the performance index replytime and other parameters monitored are shown in Table 1.

Table 1. Coefficients of correlation between response time and other parameters

2.3. Experiment II: long duration

In Experiment II, the web server was run without rejuvenation for a long time until it crashed. Each of the clients generated 270 requests per second to get an html file from the server.

Figure  4 shows that the server goes through sudden performance degradation with replytime abruptly increases after running reliably for a period of time, which is referred to as base longevity interval in [1]. The performance of the server quickly degrades and goes into a failure probable stage.

Other long duration experiments showed the same characteristic of abrupt performance degradation as in Experiment II.

Figure 4.Test results in experiment II

Other runs of this experiment also validate the existence of software aging, in which the state of software degrades with time. Instead of being a gradual process, software aging observed in our experiments is an abrupt one.

3. Modeling and Data Analysis

In Table 1, it has been noticed that average CPU load correlates strongly with system performance, so the amount of average CPU load on the server is our modeling target. The parameters in Experiment II are used as the modeling object.

3.1. Software aging process

Based on the experimental results, we conjecture that the process of software aging could be divided into four stages:

1. A highly robust stage S_r

2. A transition stage S_t

3. A failure probable stage S_p

4. A failure stage S_F

The probabilistic stage transition diagram is shown in Figure 5. Where ,  is a function of elapsed time t, denotes the transition probability from stage S_i to stage S_j.

Figure 5. Probabilistic stage transition model for system performance

When the application starts, it stays in a highly robust stage S_r, and P_rr(t), P_tr(t) and P_pt(t) is nearly 1; P_rr(t), P_tr(t) and P_pt(t) decreases with time, and P_rt(t) increases correspondingly which will lead the system into the transition stage S_t. The experimental results show that S_t  is a relatively transitory stage. The system will then enter S_p.With the increased value of P_pF(t), the system will finally crashes.

3.2. Modeling

Since loadavg correlates strongly with response time, the variable loadavg can be deemed as an indicator of system stage. To simplify the process of modeling, two constants A and B are used in the paper to partition different system stages: when loadavg is below A, the system is at stage S_r; when loadavg is above B, the system stage is S_p; when loadavg is between A and B, the system is at a transition stage S_t.

To describe resource usage in different stages, a 3-threshold model is constructed:

   (1)

The delay parameter d is determined by investigating the Partial Autocorrelation Function (PACF) of loadavg. The lag at which PACF has the peak value is selected as d.

The optimal values of A, B and are selected by varying A and B over a selected range. For each pair of A and B, the optimum order  p_j for the j^th submodel corresponds to the value p that yields the minimum value for the Akaike  Information    Criteria (AIC) statistics AIC(p_j) [6].The total AIC is computed by . This process is repeated for all subregions for each pair of A and B. The optimal values of A, B and are those that yield the minimum AIC_total. The process resulted in d=1, A=5,B=16, p₁ = p₂ = p₃=2.

Figure 6. Measured and modeled average CPU load in Experiment II

3.3. Comparison with AR model

From Table 2 one can see that TAR model is superior to AR model in every subregion. The total AIC of TAR model is also lower than that of AR model.

In the second submodel, the square sum of residues is reduced about 67% by TAR. Since the second region mostly consists of points that change rapidly, the capability of TAR models to respond more rapidly to sudden changes than AR models is validated.

 The advantage of TAR model over AR model lies in that the AR modeling in threshold models is more accurate in each region than a single AR model. Furthermore, the threshold AR model is more capable to respond rapidly to sudden changes.

Table 2.Comparison of AR and TAR model