| Real-Time TV Program Distribution and Storage Server with Keyword Access Capability | ||
|---|---|---|
| Prev | Chapter 3. Architecture of the SBSVS | Next |
In scheduler-driven operations, the SBSVS performs actions on database server's metadata and on push server's stored video. In this mode, the requirements for preserving data consistency on faults is very important. The system should operate without system administrator monitoring. When a serious fault happens, it should be logged for later inspection. But after that the system should continue its operation and preserve data consistency. This chapter describes the way possible faults are handled.
We must consider those system faults: database server crashes, one of acquisition servers crashes, one of push server crashes. The crash may happens when this particular component is involved into the recording cycle.
SBSVS is designed the way that the maintainance of system is delegated to the database server, which is written on Java and is not involved in real-time data transfer. This approach is proven to be reliable and is similar to the middleware approach. In middleware, the business logic is implemented in the layer separated from expensive and reliable back-end and simple and cheap client. When a component at the middle layer crashes, the whole system continues to operate.
In SBSVS the back-end database server was tested to run for several weeks without the crash. But the possibility of the crash should not be neglected in the design. In current implementation, it is however assumed that after the crash, the database server should be restarted during several hours, otherwise those push servers that continue the recording will fill the whole disk.
The case of manual recording is different, because all problems are logged back to the client and user may perform the immediate action on it.
The main approach to increase the fault tolerance in scheduled operations is to preserve the database consistency across every atomic transaction. The main criteria is that the shutdown of any component of the system or the failure of any database transaction should preserve the system state for recovery or rollback.
When the scheduled recording starts, the database records for new movie and new movie files are created.
Fig. 3.3 the database relation for the movie
CREATE TABLE movies (
// -- Title must be unique
title text not null,
// -- uniquely generated in sequence and used as
// -- foreign key in moviefiles and captions
id int not null,
// -- comments
description text,
starttm timestamp not null,
stoptm timestamp,
// -- if 'expires' field is not null, database server is
// -- responsible to delete the movie
expires timestamp,
// -- when the movie is already recorded, there is no need to know
// -- the channel id - threat it as the garbage, but when recording,
// -- this is very important for consistency
channel_id int,
constraint moviekey primary key (id)
);
|
The "stoptm" field is set to null to indicate that the recording is not terminated yet. The "channel_id" field indicates what channel is recording. When the recording ends, the "stoptm" field is updated with the movie end's timestamp. This information is sufficient to restore the system recording state after the crash of the database server. Each time the database server starts, it read the list of unfinished movies ( with "stoptm" equal null ). Those movies are likely to be recording. If more than one not terminated movie exist for the same channel, this is considered as a severe error. The recording is terminated and the database is updated. Then the database server checks the schedule relation to determine when the recording of each of movies must end and terminates all expired recordings. Then database server checks for all expired movies and removes them.
If any component other than database server crashes, the database server is responsible for terminating the recording associated with this component and update the database accordingly. The acquisition server keeps the connection with database/metadata server open all the time and informs it about any failure with recording. Acquisition server maintains the permanent socket connection with each of its push servers and reports the crash of any of them to the database server. And finally, if the connection with acquisition server is lost, the database server terminates all recordings associated with this channel.