Tips & Tricks
Technique : PC MusicianCheaper and larger-capacity hard drives are becoming available all the time. Martin Walker takes a look at ways for PC musicians to manage this ever-expanding audio storage space.
Only a few years ago 1Gb hard drives were the norm, and adding a 2Gb or 4Gb drive seemed like an unnecessary luxury. Nowadays it's getting increasingly difficult to buy a drive with a capacity less than 8Gb, and models with drive capacity exceeding 73Gb are beginning to appear. With such vast amounts of storage available at ever more reasonable prices, it's easy to assume that storage problems are at an end. However, having acres of storage space can create its own set of problems. If you treat your drive like one huge toy box full of operating system, applications and data stored at random, it can become increasingly difficult to find particular files as it fills up.
It's certainly possible to organise the contents of a hard drive more effectively -- Windows already helps in a small way by placing most of its own files inside the Windows folder, and applications normally offer to install themselves inside a unique sub-folder within the 'Program Files' folder. However, your own data will often still be scattered about inside the application folders, and thus be awkward to find and back up en masse. Many musicians avoid the issue and don't bother to back up their data at all, but then they face the possibility that a bad Windows crash may not only corrupt their Registry or system files, but also destroy some of their irreplaceable sounds and songs at the same time.
For the musician who regularly creates lots of huge digital audio files, it can also really dent performance to have them scattered around a huge drive, crammed in between lots of smaller files and possibly fragmented. Of course you can regularly defragment your drive to increase performance, but this could take a long time if you have a single vast drive, and your audio files may still end up in distant physical locations.
The answer to these problems is to separate Windows, your applications, and your data, so that backups are much easier to implement. This means that, even if the worst happens and you need to completely reinstall Windows, your own data is still safely tucked away elsewhere. Moreover, this approach might also allow you to increase overall performance by organising your files more effectively.
Multiple Partitions
Hard drives can be split into a number of partitions such that each becomes an individual notional drive and is given its own drive letter, such as C, D, or E. As far as Windows is concerned, the physical drive acts just like several individual ones. However, splitting an existing drive into several partitions can be tortuous work -- you need to back up any useful existing data, delete the current partition (using the FDISK command in DOS), create several new partitions (again using FDISK) and format each of them (using DOS's FORMAT command) and then finally reinstate Windows, all of your applications, and your backed-up data. Thankfully, several utility programs are now available which not only automate this convoluted procedure, but also let you add, merge and resize partitions without disturbing the existing contents of your hard drive. The most well known of these is Powerquest's Partition Magic, now at version 5.01, though some users may still have the now-discontinued Quarterdeck Partition-It.
By creating a separate partition for your audio data, you not only make it easier to keep it defragmented, but on a huge drive you can take this further by giving each major project or album its own partition. This will ensure that all the files are in a similar area on the hard drive, which will minimise the seek time of the read/write heads, and therefore increase track count. Even if you suffer from fragmentation during a recording, the read/write heads still only move within that partition, rather than jumping anywhere within a single vast audio drive to find more space.
Backing up can be far easier if you keep all your data in its own separate partition, since you can simply drag its entire contents across to a separate backup partition on the same drive using Windows Explorer. This is an ideal approach if your MIDI + Audio sequencer doesn't have a multiple undo facility -- you can mess around with your music to your heart's content without having to worry about losing something vital. Ideally you should also back up periodically onto a completely separate drive, such as a Zip, Jaz or CD-RW, for maximum security.
Another beauty of creating multiple partitions is that you can install several different operating systems, one into each, and then choose which one you want to run each time you boot up your PC. You could install Windows 95 and 98 on the same PC, or Windows 98 and BeOS, Windows 95 and NT4 -- there are all manner of possible permutations, and programs such as Partition Magic provide specific details on how to implement each one. If you can't avoid running games on your music computer, you can create a second Windows installation on a different partition optimised for this use, without compromising the efficiency of your installation for music.
Multiple Drivers
Many specialist suppliers of music PCs routinely install two drives -- one for Windows and applications, and the other strictly for audio data. This has the advantage that even if your Windows drive emits a puff of blue smoke and dies, your songs will still be safe. In addition, it also allows you to write audio data starting at the very outside of the dedicated audio drive, which provides the fastest data transfer rate. This is because read/write tracks on a hard disk are arranged in concentric circles and because the longer outer rings contain more sectors. At a fixed spin speed more sectors can be read in a single revolution and, depending on the drive, this can mean a difference in the maximum transfer rate of as much as 50 percent between data stored on the outside and inside rings.
Expert opinion is somewhat divided about the relative merits of placing audio data on a separate partition or drive. Intuitively, one might think that a separate drive would always provide better performance, but once you have launched your audio application (assuming you have enough RAM to avoid using the Windows Swap file -- see the Temporary Disk Files section below) your PC may have no need to access the hard drive containing either Windows or its applications. In this case only the audio data area is being accessed anyway, so surely it oughtn't to matter whether it's on a separate drive or simply on another partition on the same drive?
As usual, things aren't quite that simple. For a start, few applications load in every chunk of code they need from the hard drive when you first launch them -- if you keep an eye on your hard disk activity LED while using your MIDI + Audio sequencer then the chances are that you'll see it light up whenever you open a new dialogue window for the first time, although probably not when you access the same feature again. However, you can avoid most application disk accesses during critical operations like recording and mixdown by not
FATs And Small
Hard drives can be formatted in various ways. For Windows 95 and 98, the two main choices are FAT16 and FAT32, where FAT stands for File Allocation Table -- a system for storing files. The latter is more efficient with large drives, since it allows various choices to be made about cluster size. Clusters are the smallest unit of storage on a drive and can vary greatly in size, but the most typical sizes are 4K, 8K, 16K, and 32K. Using the older FAT16 system, the cluster size was automatically determined by the capacity of the drive, and the maximum supported drive size was 2Gb. Typical drive sizes at the time when Windows 95 exclusively used FAT16 varied between 1Gb and 2Gb, and this range automatically used a cluster size of 32K.
Because an individual cluster cannot hold more than one file or portion of a file, small files of only a few hundred bytes long will still use the whole cluster, wasting most of the available space. When you need to store a huge number of tiny files, as Windows does, this can leave a significant amount of space used, and when a large drive is formatted using FAT16 it's not unusual to find that hundreds of megabytes are wasted overall.
FAT32 was introduced in the OEM Release 2 of Windows 95, and is also used by both Windows 98 and 2000. It can cope with drives beyond 32Gb in size, and also overcomes wastage of space by letting you choose a suitable cluster size whatever the size of your drive. For drives or partitions containing Windows itself, a small cluster size of 4K is usually recommended, to make maximum use of space, and you should also format a drive with a small cluster size for storing any small audio files, such as banks of plug-in presets and collections of synth SysEx data.
Microsoft claim that the average user can expect to see an increase in hard disk space of 25 percent or more by reformatting a typical FAT16-formatted hard drive using FAT32. However, some people still format their Windows partition or drive using FAT16, which (being simpler) has slightly less overhead. Although this is likely to waste several hundred megabytes of drive space in a typical 2Gb partition, it may let your PC boot up slightly more quickly, and be a little more snappy when loading applications.
Audio drives and partitions should always be formatted using FAT32, since they will be larger than the maximum 2Gb supported by FAT16. They should also use the largest available cluster size of 32K. This is mainly because musicians tend to have far fewer audio files, and each of them is huge -- wasted space is therefore likely to be far less, and by dividing your drive into a smaller number of larger clusters it will be less sensitive to fragmentation.
opening any new windows, and this should help get maximum audio performance from any system running on a single drive.
The important thing is to make sure your audio data is as close as possible to the outside of the drive, to give it the highest possible sustained data transfer rate, and this can be done in several ways, depending on how many hard drives you have.
Getting Down To Brass Tacks
I'm not going to provide any step-by-step guides to partitioning, for several reasons. Firstly, there are several partition utilities, all of which operate in subtly different ways, and secondly, although the process is fairly easy to follow, there may be a large number of individual steps involved. Finally, there are just so many possible scenarios that it would impossible to cover them all in the space available to me here. If you do need a step-by-step guide, you might be lucky enough to find one on your software manufacturer's web site, such as those at www.powerquest.com, but this is not necessarily the norm.
If you do have a single large drive, the most sensible approach is to split it into at least two partitions and preferably three -- you ought at least to have a separate partition for your audio data, but separate partitions for Windows and for your applications are also a good idea. Some experts suggest that you also make an absolutely tiny outermost partition C (only 50Mb or so in size) just sufficient to contain the files required to boot up your PC, and then that the remainder is divided between a large audio data partition D and an third partition E of about 2Gb for housing both Windows and applications.
However, my own measurements suggest that allocating several hundred megabytes for the outermost partition, rather than 50, will only reduce the maximum speed of the audio partition by a couple of percent yet will provide quite enough space to install Windows, especially if you don't install all the questionable accessories such as the 30Mb of 'Desktop Themes'. With Windows 98 you can even uninstall unnecessary options that are already stored on your hard drive, by using the Windows Setup page in Add/Remove Programs. By installing Windows on partition C, you leave partition E entirely free for your applications.
The beauty of using a software utility to configure your partitions is that you don't necessarily need to know in advance exactly how much space is required for either Windows or its applications -- you can install Windows in a generously sized partition, and then shrink it later on when you see how much empty space is left. Similarly you can alter the size of the E partition to accomodate new applications, allowing your single drive to adapt to changing circumstances.
Spreading The Load
Many people upgrade their drives to cope with the demands of audio work, yet retain the older ones to use for data storage or for running their operating system. However, if you do want to replace the hard drive currently containing Windows and applications with a faster or larger model, you don't necessarily have to start from scratch. There are utility programs which let you copy all existing data onto your new drive, after which you can simply swap over their Master/Slave status by switching the cables. When you next switch on your PC your new drive will boot up as C, and you can then reformat your old drive for other uses. However, if you have been using your older drive for several years then you will probably benefit from the chance to perform a clean install of Windows on the new drive, and then reinstall your applications with a little bit more thought as to their destinations.
If you have several drives then your options start to multiply. I currently have three hard drives installed in my PC -- a 2Gb C drive containing Windows 98 and applications, a 2.5Gb drive partitioned into 500Mb Windows 95 partition D and 2Gb partition E for general documents, application update files downloaded from the Internet, and plug-in and SysEx banks, and a 4.5Gb 10,000rpm Ultra Wide SCSI drive for audio recording. By today's standards these drives are all on the small side, but many musicians find themselves in a similar situation -- there's absolutely no point in discarding a working hard drive unless it's noisy, generates too much heat, or occupies too much space in your PC's case.
Temporary Disk Files
Your PC regularly needs to store various temporary files, which can make overall performance worse if not placed on the most suitable drive or partition. As I explained last month, Windows shunts data that cannot fit into your RAM into its Swap file, so that you seem to have almost unlimited 'virtual' memory. This enables you to load and run lots more simultaneous applications than would actually fit into RAM, but of course when you move between these applications the active one must always be in RAM.
All this data shunting to and from the hard drive slows down general operation, but can be completely avoided by installing enough RAM to hold everything you need simultaneously. You can find out your current 'Swap file in use' size at any moment by running Microsoft's System Monitor utility, and you should ideally add enough RAM such that this always remains at zero when you are simultaneously running all the applications you need. It is especially important for musicians to do this, since avoiding Swap file activity will probably increase the maximum number of tracks that you can run simultaneously, and certainly the number of SoundFonts or other sample banks that can be accessed in RAM.
Having said that, it is still sensible to make sure that the Swap file can operate as fast as possible if it does have to be used, and this means choosing a suitable drive, partition, and even position within that partition to situate it. As explained previously, the fastest area on any drive or partition is on the outside, and if you set up a permanent swap file (I explained how to do this last month) then you can use the Norton Utilities Speed Disk application to move it to this position for you, as well as defragmenting it at the same time. Once it's in a single contiguous chunk, a permanent swap file will never become fragmented again. Speed Disk will place the most frequently accessed files next, and infrequently accessed files to the inside of the drive, leaving the files most frequently modified in the middle. As you can see in the screenshot above, this normally means that the outside of the drive also stays fairly defragmented.
Many applications also create temporary disk files, especially when 'undo' facilities are provided. With the huge temporary files that may be generated during the course of editing audio tracks, it is therefore extremely important that any location options provided in the host application are set up carefully. The most basic consideration for this type of temporary file is available disk space, since this will affect the number of reversions that you can manage. If you are working with 10 minute stereo CD tracks these may each be 100Mb, so carrying out 10 edits on the whole file will generate 1Gb worth of temporary files. If you suddenly receive an error message along the lines 'Can't Undo' then you are likely to have run out of disk space. Some applications let you specify the number of Undos available, and if you regularly work with huge files then it may be worth limiting this.
Housekeeping
Periodically it's worth making sure that you clear out any accumulation of temporary files that haven't been automatically deleted by their host application. The biggest container of these is the Recycle Bin, which holds deleted files until it is emptied. Sometimes you may change your mind about a deletion, so this is a useful function, but you can empty it at any time by right-clicking on the Recycle Bin desktop icon, and selecting 'Empty Recycle Bin'. More comprehensive for Windows 98 users is the Disk Cleanup utility, which lets you delete temporary Internet files, as well as files in the Windows Temp folder. These are sometimes left behind after program installs, particularly if your PC has crashed. One of my drives once gave error messages that it had run out of space, and the reason turned out to be a huge temporary audio file created by a rogue application. It's normally safe to delete anything that is stored in the Windows Temp folder, as well as any other files you find on your hard drive that end in .tmp, .old, .bak, or .000, .001, and so on.
However, Norton Utilities has proved to be the most comprehensive housekeeping utility in my collection. Its Disk Doctor is speedier and more comprehensive than the Windows Scandisk, Speed Disk optimises your hard drive Swap file and contents far quicker than Microsoft's Disk Defragmenter, and the Optimisation Wizard nicely reorganises the Registry to reduce its size and improve performance.
However, as soon as you exit your application it should clean up all of its temporary files, so your disk space will reappear.
If you have several drives then choosing a faster one for temporary storage can speed up editing. However, since your audio files are hopefully stored on your fastest drive, this has to be offset against the fact that storing temporary files within another partition on the same physical drive as your audio can in many cases halve your audio drive's performance -- this is because the read/write heads have to continually leap backwards and forwards between completely different areas of the drive.
For the same reason, situating your Windows Swap file on a different drive from your audio data can improve performance, but putting it on a completely different partition of the same drive may slow things down. If you have an old drive spare you might even want to use it solely for the Swap file, although this is perhaps going a little far. It's certainly not worth buying a new drive for this purpose -- it's cheaper and more effective to install more RAM and obviate the need for a swap file completely.
In general, you should read your audio application manuals carefully, and follow their specific advice regarding the temporary files they create. For instance, Sound Forge and Wavelab both let you define a special folder for temporary storage, which can make it easier to track down and delete stray files. Cool Edit Pro lets you specify primary and secondary temporary directories, but recommends that they are on separate physical drives.
Adding Extra Drives
If you are thinking of adding a second or third drive, you'll need to know where to stick it (in the tasteful sense, of course). Nearly all motherboards have two IDE channels where you can attach devices such as hard drives, CD drives, or removable-disk drives. Each channel can support two devices, but one must be designated Master, and the other as Slave, according to the positions of jumpers on the back of each device. This allows a total of four devices to be connected simultaneously, but only one device on each channel can be accessed at any time (either the Master or the Slave).
Although I've already covered these issues in some depth in March 2000's PC Musician, I've carried out some further research on the best choices for various systems. Unless you have an all-SCSI system, the Primary Master is always the Windows boot drive, so if you have two EIDE devices then the second should always be connected as Secondary Master, to keep it on a separate IDE channel. This applies either if you have a single hard drive and CD drive combination, or if you have a second audio hard drive. If only one device is connected to each cable, it's also good practice to attach it to the connector at the far end of the cable, to minimise the possibility of signal reflections.
The situation becomes more complicated when you add a third IDE device, such as with a system incorporating a Windows hard drive, an audio hard drive, and a CD-R drive. The best solution here seems to be to place the audio drive as Secondary Master, and then the CD-R drive as Primary Slave -- this ensures that your audio drive and CD-R are on different IDE channels when you want to burn audio CDs, and also that your audio drive can't be disturbed by the CD-R drive. I've certainly come across recommendations that you make the CD-R a Secondary Slave, but musicians don't want any possible compromise with their audio drives.
Adding a fourth IDE device can certainly be done, but few people I've spoken to recommended it. If you need more than three IDE devices then it would be better to buy a separate PCI controller expansion card and add more independent IDE channels to your system, or to think about investing in a SCSI system, where many more devices can be accessed simultaneously.
UDMA66 Pitfalls
Finally, anyone thinking of adding another hard drive to their PC is likely at least to consider one of the recent UDMA66 drives. I discussed these in the PC Musician feature in SOS January 2000, and explained why you
A Timely Warning
NEVER modify partitions unless you have a full backup of the data contained on them -- 99 percent of the time you will have no problems during the modifications, but if you experience a power cut or, as happened to me last month, your hard drive exhibits a fault while changes are being made to it, it is possible to lose its entire data contents in a fraction of a second. Worse still, unlike typical hard drive errors that you may be able to work around or repair, crashes that occur during major reorganisations of your data will probably leave it unrecoverable by all but the most experienced and dedicated sleuth.
won't get a noticeable improvement in hard drive speed simply by changing from a UDMA33 to a UDMA66 drive. However, hard drive performance is always being improved, so recent drive models are likely to have better performance than older ones anyway. Also, you have little choice in the matter, since nearly all currently available IDE drives are now UDMA66, so here are a few reminders and additional warnings.
If you want to install a new UDMA66 drive then you should ideally have either a motherboard designed to support this, or add a separate PCI controller board specifically designed for UDMA66 devices. It is also perfectly possible to use a UDMA66 drive in a PC that only supports UDMA33 mode -- this should have a negligible effect on its performance. Unfortunately some people have had problems doing this, largely due to compatibility issues with some older motherboards whose BIOS is dated 28/10/98 and older. The motherboard attempts to set itself to an unsupported mode when it can't recognise the drive, and may hang when you boot your PC, or end up in DOS compatibility mode when viewed in the Performance page of the Control Panel's System menu.
To overcome this you need to force the drive into UDMA33 mode by hand, using a utility downloaded from the drive manufacturer's web site. If your PC is two or more years old it will be worth a trip to the motherboard manufacturer's web site as well to see if you need a BIOS update to correctly recognise and reconfigure a UDMA66 drive -- if you have any doubts about this then make sure before you buy a drive that its manufacturer has a suitable utility available.
Even if your motherboard already supports UDMA66, you may also need to buy an 80-way ribbon cable to connect your new drive, instead of the 40-way type used by UDMA33 drives. The extra 40 conductors are all used for shielding, and are sandwiched between the 40 data lines to reduce crosstalk and increase data integrity with the double-speed timing signals, although the 80-pin connectors are identical to the 40-pin variety. Both the PC and hard drive can detect the presence of an 80-pin cable, and will only enable UDMA66 mode if it is found. Conversely, if you only have a UDMA33 system then you don't need an 80-way cable, even with a UDMA66 drive.
By the way, all of the new Apple iMac systems support UDMA33, and the G4 computers support UDMA66. It's funny how more and more PC and Mac peripherals are starting to converge..
Published in SOS August 2000
No comments:
Post a Comment