[vdr] Re: two or more harddrives and performance

[Jon Burgess <mplayer@jburgess.uklinux.net>]

Manfred Schmid wrote:

> hello,
> i am using vdr with 1 hardrive, this has a performace of about 17mb/s. With 
> more than 3 recordings the vdr will have many dropouts - i think this is 
> cause of performance of hardrive. 
I had been noticing a few performance anomalies when accessing files 
created by simultaneous recordings. I noticed  particular problems when 
I recorded a radio program and a TV program simultaneously. When I later 
tried to read the radio file I found that the drive maxed out at a read 
speed of only 1MB/s (the drive which normally manages 30MB/s on large 
reads!).

In order to try and narrow down the cause I wrote a benchmark program to 
emulate the kind of simultaneous streaming writes performed by VDR. The 
benchmark opens a number of files and then loops around writing a small 
amount of data to each file in turn. I don't have the source with me at 
the moment, I'll will post a copy for reference later.

What I found was that the performance depends a lot on the filesystem 
used. I don't have the results file with me either, the information 
below is all from memory, so please forgive me if I post some figures 
later which don't quite match these, but it should give you an idea of 
the kind of differences that i'm seeing.

The performance of the drive when reading a single big file was around 
30MB/s
Then I tested the same drive by writing 2 files simultaneously (4kB to 
each file in turn) and then measuring the speed that the file could be 
read back from the drive.
- The worst performing filesystem was EXT3 (I think about 6MB/s)
- Then EXT2 and Reiserfs both  ~12MB/s.
- The best performing were XFS and JFS both ~27MB/s, a near perfect result.

I believe that the cause for these massive performance differences seems 
to be that the block allocation strategies of the filesystems start to 
degenerate when they have multiple simultaneous small writes. The result 
is that the blocks on the disks for the different files get interleaved, 
fragmenting all the files.

The ~50% performance of ext2 and reiserfs are probably explained by this 
interleaving of the disk blocks of two files on the disk.

EXT3 seems to have even bigger trouble, I think that the journalling 
information ends up fragmenting the file even more.

I believe that XFS and JFS must be intelligently allocating the blocks 
on the disk, aggregating the small writes together so that the disk can 
read the data in much larger, unfragmented, amounts.

The Radio+TV case is an extreme example because the data rate is approx 
20 : 1 (500kBps:24kBps). When these files are interleaved on the disk 
the result is the drive performance being reduced to ~1/20th of the 
unfragmented read rate.

VDR could help the situation by buffering up the writes into much larger 
blocks. Writing the data in blocks of around 256kB - 1MB seem to almost 
completely avoid the slowdown with ext3. I took a quick look into the 
file writing code and there doesn't appear to be an easy way to do this.

   Jon




--
Info:
To unsubscribe send a mail to ecartis@linuxtv.org with "unsubscribe vdr" as subject.