Database server

Database server is the "brain" of the whole application. It is connected to the relational database, PostgreSql in our case. It stores there all information about movies, their locations, closed captioned text and all the system configuration. Database server do not directly participates in mpeg data traffic, but it is the only one component which have the complete information on the video server topology. Next, database server is the only component which is single in the installation - all other components may duplicate.

Architecture

Database server is the Java application, connected to relational database server through JDBC driver. No direct access to the relational database is allowed to any other components of the video server, so we may say that database server present a kind of middleware service to everybody else.

So, here and then the term Database server refers to the Java application, part of Video Server project, rather than to the relational database.

Database server is the multithreaded application. All threads performs interactive services to other components, no real-time tasks are performed in this application.

Database schema

Database is initialized by the script "tools/ClearDB.source". The most recent database schema is there. Schema have three tables - list of movies, list of movie files and captions.

CREATE TABLE movies (                                      (1)
	//   -- Title must be unique
	title   	text not null,
	// -- uniquely generated in sequence and used as 
	// -- foreign key in moviefiles and captions
	id      	int not null,                                    (2)
	// -- comments
	description     text,
	starttm         timestamp not null,
	stoptm          timestamp,                                (3)
	// -- if 'expires' field is not null, database server is 
	// -- responsible to delete the movie
	expires         timestamp,
	// -- when the movie is already recorded, there is no need to know
	// -- the channel id - threat it as the garbage, but when recording,
	// -- this is very important for concistency
	channel_id	int,                                           (4)
	constraint moviekey primary key (id)
	);

CREATE TABLE moviefiles (                                  (5)
	//  -- refers to movies
	//  -- it may be several moviefiles with this id
	id      	int not null,
	// -- uniquely identify the push server which holds this movie
	pushserv_id int not null,
        // -- store the relative path of file, not from root
	mpegfile        varchar(255) not null,
	starttm         timestamp not null,
	stoptm          timestamp,                                (6)
	foreign key(id) references movies
	);

CREATE TABLE captions (                                    (7)
	// -- movie id
	id      	int not null, 
	txt     	text not null,
	captime 	timestamp not null,
	foreign key(id) references movies
	);
       

(1)
Table holds the list of movies, id is the primary key.
(2)
New ID is selected in sequence in the method DatabaseConnection.add_new_movie().
(3)
All fields except stoptm are set in the call DatabaseConnection.add_new_movie(). The stoptm field is set when the recording is stopped, at DatabaseConnection.mark_movie_stop_time() or when recalculating of movie time is requested explicitly, in DatabaseConnection.fix_duration().
(4)
The optional "expires" field provides the scheduler with the capability to recycle movies to provide empty space.
(5)
Table holds all mpeg data files, associated with this movie. It maybe several movie files with the same ID, and it may be some different push servers that store this file. File names of the same movie may be and may be not different on different push server storages.
(6)
All fields except stoptm timestamp are assigned in method DatabaseConnection.add_movie_file(). stoptm is assigned when this particular movie recordings ends, in method DatabaseConnection.mark_movie_stop_time(). It is possible to have a gap between the end of one moviefile and the beginning of the next one in the same movie.
(7)
Stores closed captioned text for all movies

Why movie may consists of several movie files?

When the recording starts, new entities are inserted both into table movies and table moviefiles. When recording stops, both entities are updated to reflect the correct time when the recording ends. Note, that the recording of this particular movie may happens on several push servers simultaneously, so several entities of moviefiles may be inserted, but they are assigned to different push server id's, so to simplify things we can say that this movie still consists of the single movie file. However, the database server supports the append operation, which lets to append one movie to another. Effectively, the title "movies" entity for the second movie is destroyed, and its files and captions are appended to the first movie. In that case we may have a movie which consists from several files, with possible time gap between.

Interface and protocol

On startup, each of the clients and acquisition servers open the permanent interface connection to the database server. This connection serves to send commands and to receive answers from the server. Database server starts the new thread dedicated to every connection. Synchronization mechanisms are used when threads concurrently access shared resources. If connection is broken, database server assumes that the peer application is down, so it terminates this thread.

The protocol to access the database server interface is text-based. So, it is possible to connect to the control port of the server via telnet and to type all commands from keyboard. The reply is text as well. Another way to exploit the text-based protocol is to write scripts on perl or python languages to have a quick and easy way to implement additional database management functionality.

Some key protocol commands

Database server protocol is described in file TextProtocol.java. Here is only the brief overview:

Session control: kill, superuser, debug, help, saveproperties, listproperties, setproperty

Access the movie database and list its content: database, listtables, listmovies, listmovfiles, listcaptions, listchannels

Modify the databse: addnewmovie, addmoviefile, addcaption, addcaptionnoid updatemovieinfo, appendmovie

Record and play: startrecording, stoprecording, playmovie

Commands from acquisition server: registeracquisitionserver, setchanneldescription, requestbroadcastparameters, pushserverok, pushserverbad

All commands intended to modify the database or the server status require obtaining superuser permissions before executing.

Scheduler

The purpose of scheduler is to provide the automated operational mode to the video server. Scheduler stores triplets of times: time to start the recording, time to stop it and timeout before this recording should be deleted from the database.

Database schema

The scheduler schema is stored into the videomanager database which is initialized by the script tools/ClearManDB.source

CREATE TABLE schedule (                                    (1)
	schedule_id 	int not null,
	// -- channel ID is referred to the acquisition server ID,
	// -- not the actual TV channel number. 
	channel		int not null,
	// -- the optional ID of the real TV channel, assuming that
	// -- acquisition server may performs remote control channel
	// -- switch using infrared device
	tv_channel	int,
	// -- "code" specifies how to interpret the schedule:
	// -- "0" means this schedule is disabled ( why not ? )
	// -- "1" means this is one-time scheduler, no repeats
	// -- otherwise bit fields in range "0x4(Mon) - 0x100" are interpreted
	// -- as day-of-the-week bitmask, for example:
	// -- "0xf8" or "0x7C" means weekdays, depending on country
	// -- "0x104" or "0x180" means weekends
	code 		int not null,
	// -- do not set the schedule overnight, split it instead
	starttm         timestamp not null,                       (2)
	stoptm          timestamp not null,
	// -- name of the database where scheduled movie will be stored
	database	text not null,
	// -- pattern for the name description of this movie
	name_pattern	text not null,
	// -- time-to-live for this movie: is the offset, 
	// -- because the schedule repeats every day
	// -- only days-of-the-year, hours-of-day and minuts are meaningful
	expires		timestamp not null,
	constraint schedule_key primary key (schedule_id)
	);

CREATE TABLE schedule_pushserver (                         (3)
	// -- foreign key
	schedule_id 	int not null,
	// -- if no rows of this entity exist for this selected 
	// -- schedule id, then everybody are interested;
	// -- if it exists, then send data according to this entry
	pushserv_id 	int not null,
	foreign key(schedule_id) references schedule
	);
       

(1)
The scheduler database schema.
(2)
If this entry represents the every week repeating events, then the start and stop time are evaluated by the hour and minute; the year and day are meaningless.
(3)
This table is optional, it is needed only to specify the case when movie will be recorded only at the selected push servers. Otherwise, the movie will be recorded at all push servers, dedicated to this channel.

Implementation

Scheduler works as the separate thread which handles the sorted set of abstract class SchedulerItem. It picks up the first event from the heap, waits until the time specified in the timestamp of this event comes and call the execute method. The synchronized call to add a new event to the queue is priveded to let other threads add some new events on-the-fly.

The schedule is loaded from the database on start-up and at the start of each day.

Fault-tolerance concerning issues are discussed in the separate section, where the reliability of the whole system are discussed.