Real-Time TV Program Distribution and Storage Server with Keyword Access Capability
PrevNext

Chapter 4. Design and implementation of the Video Server

Table of Contents
Database or Metadata server
Push server
Acquisition server
Client

The main goal of the implementation was to come with a useful solution rather than to just build a working prototype. This implementation should be installed to stream TV channels in the Computer Science Dept. of SUNY at Stony Brook. The steps to do this include:

Database or Metadata server

Database server is the "brain" of the whole application. It is connected to the relational DBMS, PostgreSql in our case. It stores all information about TV programs, their locations, closed captioned text and all the system configuration. Database server does not directly participate in mpeg data traffic, but it is the only one component that has the complete information of the SBSVS topology. Next, database server is the only component which is not duplicated in the installation - all other components may duplicate.

Implementation

Database server is a Java application, connected to relational database server through JDBC driver. No direct access to the relational database is allowed for any other components of the video server, so we may say that database server presents a kind of middleware service to other SBSVS components.

The term Database server refers to the Java application, rather than to the relational database.

Database server is a multithreaded application. All threads performs interactive services to other components, no real-time tasks are performed in this application.

Fig 4.1: Threads and connections in the database server

The database connections monitor is implemented in the database server to mutually exclude the access to same database records. All threads share the pool of RDBMS connections. At the figure 4.1 the number of connections from threads to the monitor is five, but the monitor have only three connections to the RDBMS. The access to different relations may happens concurrently.

Database schema

Database is initialized by the script "tools/ClearDB.source". The most recent database schema has three tables - list of movies, list of movie files and captions and is shown in the following: 

CREATE TABLE movies (
	//   -- Title must be unique
	title   	text not null,
	// -- uniquely generated in sequence and used as 
	// -- foreign key in moviefiles and captions
	id      	int not null,
	// -- comments
	description     text,
	starttm         timestamp not null,
	stoptm          timestamp,
	// -- if 'expires' field is not null, database server is 
	// -- responsible to delete the movie
	expires         timestamp,
	// -- when the movie is already recorded, there is no need to know
	// -- the channel id - threat it as the garbage, but when recording,
	// -- this is very important for consistency
	channel_id	int,
	constraint moviekey primary key (id)
	);

CREATE TABLE moviefiles ( 
	//  -- refers to movies
	//  -- it may be several moviefiles with this id
	id      	int not null,
	// -- uniquely identify the push server which holds this movie
	pushserv_id int not null,
        // -- store the relative path of file, not from root
	mpegfile        varchar(255) not null,
	starttm         timestamp not null,
	stoptm          timestamp,
	foreign key(id) references movies
	);

CREATE TABLE captions (
	// -- movie id
	id      	int not null, 
	txt     	text not null,
	captime 	timestamp not null,
	foreign key(id) references movies
	);

The relation movies contains a list of TV programs. When a new movie is recorded, a new entity of this relation is created. The field starttm is set to the recording start time, the field stoptm remains empty until the recording terminates. If the expires field is set, the movie will be removed from the database by the scheduler.

The relation moviefiles stores the list of MPEG files of the movie identified by the ID. TV program may be duplicated on several push servers.

The relation captions holds one line of the closed captions text with associated timestamp.

Why movie may consist of several movie files?

When recording starts, new entities are inserted into table movies and table moviefiles. When recording ends, both entities are updated to reflect the correct time. Note, that the recording of this particular movie may happen on several push servers simultaneously, so several entities of moviefiles may be inserted, but they are assigned to different push server id's, so to simplify things we can say that this movie still consists of a single movie file. However, the database server supports the append operation, which allows one movie to be appended to another. Effectively, the title "movies" entity for the second movie is destroyed, and its files and captions are appended to the first movie. In that case we may have a movie which consists of several files, with possible time gap between them.

Interface and protocol

On startup, each of the clients and acquisition servers opens the permanent interface connection to the database server. This connection serves to send commands and to receive answers from the database server. Database server starts a new thread dedicated to every connection. Synchronization mechanisms are used when threads concurrently access shared resources. If a connection is broken, database server assumes that the peer application is down, so it terminates this thread.

The protocol to access the database server interface is text-based. So, it is possible to connect to the control port of the server via telnet and to type all commands from keyboard. The reply is text as well. Another way to exploit the text-based protocol is to write scripts on perl or python languages to have a quick and easy way to implement additional database management functionality.

Commands

Database server protocol is described in file TextProtocol.java. Here is only the brief overview:

Session control: kill, superuser, debug, help, saveproperties, listproperties, setproperty

Access the movie database and list its content: database, listtables, listmovies, listmovfiles, listcaptions, listchannels

Modify the databse: addnewmovie, addmoviefile, addcaption, addcaptionnoid updatemovieinfo, appendmovie

Record and play: startrecording, stoprecording

Commands from acquisition server: registeracquisitionserver, setchanneldescription, requestbroadcastparameters, pushserverok, pushserverbad

All commands intended to modify the database or the server status require obtaining superuser permissions before executing.

Example: simple broadcast session 
       cmd "listchannels"  // the client requests the list of channels
         join the channel  // no need to inform the database server about that

Example: simple playback session 
       cmd "listtables"    // get the list of metadata databases
       cmd "database"      // select one of the databases by name
       cmd "listmovies"    // get the list of movies in the database
       cmd "listcaptions"  // read the text of the selected movie
       cmd "listmovfiles"  // get the list of files to play
       cmd "playmovie"     // tell the database server to playback the movie

It is possible to telnet to the database server and to type all commands from the keyboard. This is the full transcript of the example above: 
$ telnet tv 7077
Trying 130.245.24.15...
Connected to tv.ecsl.cs.sunysb.edu.
Escape character is '^]'.
Video Server
Connection established
>>>listtables
->
video_servertest
video_dilbert
video_italy
<-
>>>database video_dilbert
>>>listmovies
->
Dilbert #1
1
1

1999-04-16 17:06:09 EDT
1999-04-16 18:27:40 EDT
Dilbert #3
4
1

1999-04-18 12:48:49 EDT
1999-04-18 14:11:11 EDT
<-
>>>listcaptions 1
->
....... a lot of output skipped
(  trilling and clucking  )
1999-04-18 12:38:41 EDT
Hey, cal, what's  the lunch special?
1999-04-18 12:39:21 EDT
Meat loaf sandwich.
1999-04-18 12:39:23 EDT
<-
>>>listmovfiles 1
->
/home/videoserver/VideoData/19990316/1706.mpg

130.245.24.15
7078
1999-04-16 17:06:09 EDT
1999-04-16 17:09:36 EDT
/home/videoserver/VideoData/19990316/1741.mpg

130.245.24.15
7078
1999-04-16 17:41:24 EDT
1999-04-16 17:48:27 EDT
..... skip
<-
>>>
    [ after that the client requests the push server 130.245.24.15,
      port 7078 to play MPEG file 1706.mpg ]
>>>kill
->
#!shutdown
<-
$
    [ end of the session ]

Scheduler

The purpose of scheduler is to provide the automated operational mode to the video server. Scheduler stores triplets of times: time to start the recording, time to stop it and timeout before this recording should be deleted from the database.

Database schema

The scheduler schema is stored into the videomanager database which is initialized by the script tools/ClearManDB.source

CREATE TABLE schedule (
	schedule_id 	int not null,
	// -- channel ID is referred to the acquisition server ID,
	// -- not the actual TV channel number. 
	channel		int not null,
	// -- the optional ID of the real TV channel, assuming that
	// -- acquisition server may performs remote control channel
	// -- switch using infrared device
	tv_channel	int,
	// -- "code" specifies how to interpret the schedule:
	// -- "0" means this schedule is disabled ( why not ? )
	// -- "1" means this is one-time scheduler, no repeats
	// -- otherwise bit fields in range "0x4(Mon) - 0x100" are interpreted
	// -- as day-of-the-week bitmask, for example:
	// -- "0xf8" or "0x7C" means weekdays, depending on country
	// -- "0x104" or "0x180" means weekends
	code 		int not null,
	// -- do not set the schedule overnight, split it instead
	starttm         timestamp not null,
	stoptm          timestamp not null,
	// -- name of the database where scheduled movie will be stored
	database	text not null,
	// -- pattern for the name description of this movie
	name_pattern	text not null,
	// -- time-to-live for this movie: is the offset, 
	// -- because the schedule repeats every day
	// -- only days-of-the-year, hours-of-day and minuts are meaningful
	expires		timestamp not null,
	constraint schedule_key primary key (schedule_id)
	);

CREATE TABLE schedule_pushserver (
	// -- foreign key
	schedule_id 	int not null,
	// -- if no rows of this entity exist for this selected 
	// -- schedule id, then everybody are interested;
	// -- if it exists, then send data according to this entry
	pushserv_id 	int not null,
	foreign key(schedule_id) references schedule
	);

The relation schedule contains the the triple of events to start recording, stop recording and to remove expired movie from the database. The code field indicates whether this is one-time event or the repeating event. For repeating events only the time of day matters, but the day and the year are ignored.

The relation schedule_pushserver is optional. It is needed only to specify the case when movie will be recorded only at the selected push servers. Otherwise, the movie will be recorded at all push servers dedicated to this channel.

Implementation

Scheduler works as a separate thread that handles the sorted set of abstract class SchedulerItem. It picks up the first event from the set, waits until the time specified in the timestamp of this event comes and call the execute method. A synchronization call to add a new event to the queue is provided to let other threads add some new events on the fly.

The schedule is loaded from the database on start-up and at the start of each day.

PrevHomeNext
Preserving Data Consistency Push server