Push Server Evaluation

To test the push server, a test client was written. This client opens as many separate connections as requested in command line, requests movies playback, receives the UDP video stream and performs measurements. The client requests several different movies from the pool to reduce the effect of disk caching.

The client performs two basic measurements. First, it checks sequence numbers and thus computes the percentage of lost packets. Second, it measure the time between each two sequent packets and computes the variance of the collected data. Variance, or the square of standard deviation, is computed as V = SD^2 = ( SUM X^2 - ( SUM X )^2 / N ) / N, where X is the timeout between one packet and the next one, N is number of packets. Results of those two measurements are tabulated and plotted at the same graph to compare and analyze.

The first measurement is made with virtual sockets, when the client is located at the same computer as the push server. Data are not loaded from the hard drive - fake data are transmitted instead. This should let us to reduce network and disk usage and to test the system with the maximum load.

Fig 5.1: Evaluation of the push server with local clients and no hard drive access. The X axis represents the amount of independent data connections, or the number of virtual clients. The left Y axis with "+" dataset represents the Variance of time intervals between packets at the client side. The right Y axis with "X" dataset represents the percentage of lost packets.

To interpret variance we should also know the mean of the intervals between packets, which is equal to 8.6 milliseconds. So the maximum variance for small number of clients is 90 milliseconds^2, or 9.5 milliseconds. This means that most of packets come with the interval less than 18 milliseconds, what is good enough for the quality of service. The maximum number of clients when no packets are lost is equal to 17.

The second measurement is made with virtual sockets, when the client is located at the same computer as the push server. This should let us to reduce network usage and to test the software and disk system with the maximum load.

Fig 5.2: Evaluation of the push server with local clients. The X axis represents the amount of independent data connections, or the number of virtual clients. The left Y axis with "+" dataset represents the Variance of time intervals between packets at the client side. The right Y axis with "X" dataset represents the percentage of lost packets.

Comparing Fig. 5.1 and Fig. 5.2 we can see that the disk is not the bottleneck in this particular case. But to simplify measurements only 3 different mpeg files was broadcasted, so in the more realistic case a slightly different result may be expected. As we can see, up to 17 clients are served well.

The third measurement is performed with the remote client. The client is connected to the gateway with 10 Mbps link, and the gateway routes the IP traffic to the 100 Mbps interface where the server is located. The gateway is a 100 MHz Pentium computer, running Linux 2.0.36.

Fig 5.3: Evaluation of the push server with remote clients. The X axis represents the amount of independent data connections, or the number of virtual clients. The left Y axis with "+" dataset represents the Variance of time intervals between packets at the client side. The right Y axis with "X" dataset represents the percentage of lost packets.

This measurements shows the practical case with the video broadcasted over the local area network with one gateway. The push server can serve up to seven clients smoothly. We can also conclude that the commodity network adds some practical limitations to the system, because in the previous test we used to broadcast 17 channels.

The next interesting point to note is the fact that in the last case the packet's interval variance is much less than in the case when video is broadcasted through the virtual socket. The possible explanation of that is the fact that in the last case push server receives all computer resources exclusively and do not share them with the client.

In all experiments the amount of lost packets is exactly zero when the number of streams is small. That differs from the test with the real client, where approximately 0.1% of packets are lost. This means that the real client sometimes fails to pick-up the UDP packet from the buffer at time. The clients networking part is located in the SIH plug-in for the MpegTV board, which is not open-sourced, so the fix of that problem will require some additional work.

Finally, we should note an very interesting effect, which appears in all experiments. The incoming packet's intervals variance starts with a relatively high value and then decrease dramatically up to the point when the maximum amount of clients is served without losing packets. For example, we can see from from the Fig. 5.3 that the variance with one client is equal to 65 milliseconds^2, but when 7 clients are served, the variance drops to 1, what is in fact below the measurement accuracy, so there is no fluctuations of intervals at the high load. The same result appears in all measurements, but in the case when the client works at the same computer as the server, the absolutely smooth transfer is not possible due to the fact that two processes share the resources.

So, if this effect is not resulted from some kind of measurement artifact or incorrect approach, it is a very interesting one. We can suppose, for example, that the kernel tries to reschedule I/O events the way to decrease the whole system load. Other explanations may come, so this effect is quite interesting for further investigations.