As usual - but perhaps more so given my ignorance - this material is unofficial. In the hope that it's helpful, buckle up for a hard ride. Also as usual, it is aimed at Windows users; Mac users should be aware that much of it needs to be adapted to their/your needs. Note, too, that though Universal Data Format (UDF) applies only to packet writing for CD-ROM, elsewhere - notably in DVD-ROM - it has other applications.
The International Standards Organization controls standards generated by industry. The standard for CD-R is numbered 9660 and covers the agreed rules for recording all sorts of CD's. Not all manufacturers of hardware and software look on those standards the same way: some violate them freely, some implement them in part, some try to adhere to them strictly. When a publisher like Adaptec tries to implement them fully and a hardware vendor does not (names omitted to avoid assigning guilt), problems arise. The user complains that Adaptec does not do something the drive manufacturer claims to do and everyone is likely to be both right and wrong. Adaptec says: the h/w does not implement the standard; the h/w maker says: it does what it needs to do; the user says: it's not my fault. There are many solutions available. Another software package may be used which is less stringent about the rules; a different device may be used which follows them more closely; the user may give up on the capability.
ISO 9660 Level 1 covers the formats we have known for a while. They are CD-DA, CD-ROM Mode 1, CD-ROM Mode 2, Mixed and CD-Xtra. In the first three, the disc may be written Disc At Once (DAO) if the hardware and software support the capability. CD-DA is covered in other pages of this primer under Audio. CD-ROM Mode 1 was designed for single-session recording; Mode 2 is slightly less efficient and was designed for multisession. In fact, either may be used for either single- or multi-session - the difference is that while only very old drives balk at multisession Mode 2 discs, some moderately old ones won't take multisession Mode 1. Both Mixed and CD-Xtra modes combine CD-DA and CD-ROM on a single disc. Simply put, CD-Xtra permits playing the audio on a conventional CD machine since the audio tracks come first. A similar format, Enhanced CD, may be thought of as CD-Xtra without its bonus folders and files.
On a conventional CD of any flavor, the first information the reader sees is the Table Of Contents (TOC). In CD-DA, the .CDA 'file' specifying each track is a fiction that a file manager (like Explorer) uses. In a single-session, closed disc, the TOC contains information which is interpreted by a DOS extension as being a directory. The rules for converting TOC information (including the fact that everything on the disc is read-only) are embedded in the extension, such as DOS's MSCDEX. On a multisession disc, the TOC for the first session tells the system where to find the next one - or whether the current one is the last, or whether the disc is now closed. DAO is necessarily single-session (which is why one cannot write DAO tracks in Mixed or CD-Xtra formats), but a multisession variant, SAO for Session At Once, is available on most drives.
In order to support UDF, the TOC - first information on the disc - had to change. As a result, MSCDEX and other extensions cannot read packet-written discs even after they are closed to ISO 9660 format; the information that the extension expects to see is simply not visible. The solution was to invent Level 3. Functionally, it looks the same as Level 1 to the user, but it requires a different extension. That extension is available in Windows 9x, Windows NT, and Apple System 8, so that they can read Level 3 discs. It is not in DOS, Windows 3.x or other systems. This has nothing to do with your hardware or application software. Someone may write a new extension, but Microsoft may not be interested and there's not much money to be made from the effort to inspire a company to do the job. If you write one, you will be much loved by many users, but probably not well paid for it.
Now comes packet writing - which has two distinct flavors. The idea behind writing packets is that if one does not worry about the TOC, one can write data to a CD-R in blocks or packets. The information that goes into each packet needs to be accessible by the writer, but not necessarily by an ordinary reader. By writing an appropriate set of functions into the heart of the OS - approximately at the level of the extensions to DOS - a vendor can provide packet capability for writers with suitable hardware and firmware. However, there is no TOC in the Level 1 or Level 3 sense; that information is on the disc in other ways. That takes extra space on the disc and means that those data must be interpreted when a UDF disc is inserted into the drive.
At one time, Sony had its own format for packet writing and provided a driver to permit a UDF disc to be used in a reader; that appears to have been dropped in favor of the standard discussed below. Roxio's DirectCD implements the UDF standard and works in Win9x and NT OSs. Other publishers offer other packet-writing (UDF) implementations, but since only one UDF implementation can be in an OS at a time, the following is restricted to DCD. If someone can provide information onother realizations, I'll be glad to append it.
In the first version(s) of UDF, only conventional, non-erasable media were considered. For them, a very efficient system could be devised. At the head of each packet is the information on the file it represents including its length (which, of course, implies where the next packet begins). Each packet contains an integral number of blocks; the only space wasted is for that header and for the leftover bytes of the last 2K block. A variable-length packet can be converted to ISO 9660 Level 3 format because all the bytes of the file are in complete blocks. Those blocks may not be consecutive, however; if one writes more than one file at a time to a packet disc, the packets may be interleaved. When the session is closed to Level 3, the TOC is written in a more or less conventional fashion, but data interpretation requires that the OS ignore the old header information. In other words, the TOC is similar to that of Level 1, but both the TOC and the data are different enough to require changes that make the old extension useless. (Note that the Level 3 TOC may be written either to close a session or to close the disc. If the session is closed, another UDF session can be started and finalized/closed; when a disc is closed, the TOC indicates it and will not permit reopening.)
A note on capacity: Variable-length packet writing is very efficient. Because of the small amount of space wasted, it is possible to write more blocks than can be accomodated by a FAT16 for a 650-MB disc. If you fill a variable-length packet disc with many files of the right size, it may not be possible to close it to Level 3. That means you won't be able to read it except in your writer - the session is still open and a CD-ROM cannot read an open session.
When UDF is applied on an erasable blank, a new option is available. In addition to being able to write variable-length packets and to erase the whole disc, you can write, read and erase individual fixed-length packets. Now the CD-RW begins to resemble a conventional floppy, where the packets are the sectors, at least from the point of view of the user. That capability is reflected in DCD 2.x and beyond. However, one problem of erasable blanks is that they support a limited number of erasures - nominally, a thousand. Another problem is that when one rewrites on any drive, data may be written over previous entries. Overwriting results in 'scrubbing' or disproportionate reuse of a single region of the disc. For fixed-length packets, the consequences include high overhead to keep track of all the blocks and inability to convert the disc to any Level of ISO 9660. So an erasable blank formatted for UDF loses 20% of its capacity to overhead, forces slow access (about the equivalent of 1x) and can't make the disc readable in a conventional player.
If you think about it for a moment, that sounds much like the case for a very large floppy - which is a good analogy to keep in mind. Both a floppy and fixed-length packets waste about a lot of raw capacity on formatting; neither can be read in a device which cannot write it; each is relatively slow; and if you have a catastrophic failure during writing, you may lose the whole disc. Of course, the UDF disc is hundreds of times bigger than a floppy, but that's a (rather important) detail.
When a write-once disc is formatted for variable-length packets, space is committed for the TOC, but no TOC is written. Nothing else of importance is done, so the process is quick. When a file is written to the disc, it becomes a set of packets headed by the information corresponding to what would be in its FAT entry and including its length so that the location of the next packet can be determined. At the same time, DCD holds in memory a virtual FAT (VFAT) which gives access to that file just as though it were on a conventional disc. Write another file and the VFAT is updated appropriately. When the disc is ejected, the VFAT is removed from memory and nothing is written to the disc.
If the session is closed when the disc is ejected, the information for the TOC is written to the disc from the VFAT but the packets themselves are not altered. The TOC is then accessed through the usual extension. Until the disc is closed, there is no usable TOC information on it - which is why it cannot be read conventionally. A disc with a closed session has a TOC in Level 3 format; that TOC may point to the place another TOC will be written or may say: this disc is closed. During the time that a packet is being prepared and written, a catastrophic system failure would cause the write to be incomplete and might make the disc useless; at all other times, even if the system is powered down with the disc in the drive, the information is safe. The laser is off, everything needed is already written and nothing can happen. Apply power, return to your OS, and the packets are read to form the VFAT.
With fixed-length packets, the process differs in some significant ways. The key fact is that when the disc is ejected the VFAT is written to the disc so that the location of the packets forming each file can be traced. Logically, one would ask: Why not write the VFAT each time the disc is written. The answer is, simply, that that would increase the number of writes to the space that it occupies and would kill the disc more quickly. The VFAT in memory becomes a FAT on the disc. Since the runout track was written, a suitable driver then gives access to the files. That driver is provided by the UDF Reader. Since a fixed-length-packet disc is always RW, that driver is only useful in a MultiRead drive or a writer capable of reading erasable media. A fixed-length-packet disc still in the drive after having been written must be ejected (and have the VFAT written) to give access to the new files. If power fails or the system is reset before ejecting the disc, the new information will be lost and the disc may be unreadable. Couple that vulnerability with the limited life of erasable media and you see why few serious users write fixed-length packets.
E-mail me at cdrecording@mrichter.com
Return to Mike's home page