Pasti format

Pasti (STX) floppy image format

This description is based mostly on Markus Fritze's article. He is floppy expert and made some protections for Atari ST in past. I added couple things to it and rearranged some parts. Made without format's author contribution, this is certainly not complete. May contain some minor errors, but intention is to help people who want to do something more with thousands of STX image available - and not just playing under emulators.

First must to say some relevant things:

Pasti STX image format is not real floppy image. Of course, I talk about images available around. Is there some non/published variant with more details, more suitable for writing onto floppies - it is not on me to talk about. I examined over 150 STX images yet. There is 2 major type: made with stock Atari ST machine and public avail. SW. Another is made with Discovery Cartridge. Main difference between is way how track image is made. 'Public' images contain track images as WD1772 floppy controller reads them. Such images are good for emulators, but not (really) for writing onto floppy. Plus, track image can not hold sector data, and it must be stored separately, so such STX images are longer - typical size is 2.1x of same format unprotected floppy image. With DC made images can hold sector data inside track image, so size is usually 1.5x of same format unprotected floppy image. More later.

Other relevant information regarding copy protections: there is fuzzy bit mask, timing info for every sector, FDC status flag etc. All it is mostly in form suitable for emulation, and not for floppy write. No wonder - Pasti is made in cowork with Steem authors, and for sure with heavy usage of Steem Debugger + Pasti Debugger in.

What are chances for having floppy write option from Pasti images one day ? How Pasti images can be used with HxC floppy emulator ? I'm not much optimistic about any of them. Even if it will be made, it will be limited. So, don't dream that you will be able to write any STX onto floppy. I'm not say that it is entirely impossible. Just conversion from current format to writeable one is very complicated. Will someone do it ever ? With HxC it is little better - some STX images already work. But only small % . Usually some simpler protections and STX images of unprotected titles. And there is danger of protections with delayed effect. It will let you play, but on some later stage will crash machine or screw screen, or similar. Track read based protections are common, and HxC can not emulate them.

The format self:

File header (16 bytes)

0x00	'R','S','Y',0	File ID
0x04	0x0300.w	Probably the version number of the format
0x06	0x0100.w	for public, or 0xCC00 for DC type
0x08	0.w	*
0x0a	tracks.b	(e.g. 80 or 160 for double sided)
0x0b	1.b	*
0x0c	0.l	*

(w-word 16 bits; b-byte; l-long 32 bits. Little endian)

I saw 3 STX file header types so far:

[1] this is by images made with public tool and stock Atari ST .

[2] images made with DC .

[3] with DC too, I guess.

After the file header one block for each track is following. Each block also starts with a 16 byte header:

Track Header (16 bytes):

0x00	l	length of the track in bytes. This value plus the current position in the file points to the following track header.
0x04	l	bytes of fuzzy sector mask in this track
0x08	w	number of sectors in the track
0x0A	w	bit mask with flags
0x0C	w	length of the track image in bytes (around 6kb is normal)
0x0E	b	track number - it is: 0 - last track on side A (usually 79 or 81), and 128 - last track on side B (128-207/9) . So, side is determined by bit 7 (clever).
0x0F	b	sometimes 0, sometimes 0xCC . DC ?

The bitmask for the flags (word at 0x0A in track header)

7 - The track image also contains a sync-offset to the beginning of the first sync marker

6 - If set the track is saved as an image with all gaps via the read track command with the sector content filled in via read sector.

5 - with PASTI 0.4b always set, if Bit 0 is set (= track protected)

4 - unused

3 - unused

2 - unused

1 - unused

0 - If set the track is protected or with a non-standard sectorsize (!= 512 bytes)

Above description may little confuse. In practice, there are some typical values:

1. 0xC1 - made with DC - track is not saved with WD1772 read track command, but with DC. Typical track length is about 7KB. Sector datas are inside track data.
2. 0x61 : made with public tool. There is track image, as WD1772 read track command readed it. There are all sector datas too. It means that total track data is about 10KB. But in practice, track image is needed only for couple tracks (protections) or for none. So, there is overhead, caused by reason that images are made usually by people not able to determine what tracks are protected.
3. 0x21 : there are only sector datas. No track dump. Usually by DC type. By public not - all tracks are usually with flag 0x61.
   Empty tracks are with flag 0x21 and 0 sectors. Actually, then there is only track header present.
4. If Bit 0 is clear, the track is unprotected and the number of sectors as stored in the track block header are simply saved as 512 byte blocks directly after the header. The track is therefore 16 + 512 * sectors bytes long. - I never saw such.
5. If Bit 6 and 7 are clear, there is no track image and the sectors are stored just by order end length - just like the fuzzy sector mask, except this time for all sectors. Warning: sectors can be missing, if only a header was read, but the data was missing. The FDC status has Bit 4 (Record not found) set in this case.
6. If Bit 6 is set and 7 is clear, the track image is written with a track image header and the sectors follow the track image in the usually order and length. The track image header is just a word (little endian, as usual) that contains the length of the track image.
7. Bit Bit 6 and 7 is set, the track image is written with a track image header that has two words: the first one is a byte offset to the first 0xA1 address mark sync, the second word is the length of the track image in bytes. DC type.
8. If Bit 7 is set, the sector data is usually merged inside the track image, the sector offsets then point inside the actual track image. However, it is possible for single sectors to have an offset _behind_ the track image, if PASTI couldn't map them correctly into the track image. A software should always read the sectors based on the sector offset in the sector header. DC type.
9. If Bit 0 is set, the track header is first followed with the sector headers. One for each sector in the track, again 16 bytes per header.

Sector Header (16 bytes):

0x00	l	sector offset inside the track data
0x04	w	position of the sector header relative to the beginning of the track as a time stamp. The position timing is depended on the RPM speed of the drive, but for a ~6250 byte track the value for a full rotation is around 50000 (for 300 rpm). This allows software to simulate the time it takes to find the sector and read it, something certain copy protection schemes test.
0x06	w	read timing of the sector data, certain copy protections write sectors so, that it takes longer to read them. The protection measures the time to read the sector. 16384 is the default value, larger values mean that it takes longer to read the sector than normal. However if the measured value by PASTI is within 2% of this value (16384 +/- 320), the value written to the stx file is 0 as a sign for a stable read speed.
0x08	b	Track number from the address block identifying the sector (typically 0..79)
0x09	b	above 2 may be false, as part of the protection (Gold of the Aztecs) .
0x0a	b	Sector from the address block identifying the sector (typically 1..9)
0x0b	b	2=512 or 3=1024 bytes)
0x0c	b	First byte of the CRC over the address block
0x0d	b	Second byte of the CRC over the address block
0x0e	b	FDC status register after reading the sector
0x0f	b	Sector flags, always 0x00

The CRC is a CCITT CRC16 calculated by the FDC. It is initialized with 0xFFFF and includes the sync marks, therefore typically 0xA1,0xA1,0xA1,0xFE plus the 4 bytes from the address block.

The FDC status is typically 0x00. Also common is 0x10 for a sector not found and 0x08 for a CRC error when reading. Bit 7 has a special meaning: it hints that the sector has fuzzy bits and the fuzzy sector mask should be used. If Bit 7 is set, Bit 3 is usually also set, because fuzzy bits will trigger a CRC error inside the sector.

After the sector headers the fuzzy sector mask is following, if the value is != 0 in the track header. The mask identifies the bits that are random in a given sector. The size of the fuzzy sector mask depends on the sector size (from the header) and the number of fuzzy sectors in the track itself. The fuzzy sector mask has bits set for every bit in in the sector, that is _not_ random. Therefore a byte generated with the mask could be calculated like this:

fdcSectorByte[byteOffset] = (sectorData[byteOffset] & fuzzyMask[byteOffset]) | (rand() & ~fuzzyMask[byteOffset])

After the fuzzy sector mask, the sectors are written to the file. If neither Bit 6 or 7 in the track flags is set, only the content of the sectors is written. Again with the sector size in the header taking into consideration. These tracks can be also easily recognized by sector offsets that are always a power of 2 (typically in 512 byte increments), starting at 0.

Following the fuzzy sector mask is the track data area. This area can contain either the sectors by itself, or the complete track image with the sectors embedded, or the complete track with the sectors separate (following the track image). How they are stored is stored in Bit 6 and 7 in the track flags.

The track order:

Sometimes tracks are imaged in sidewise order: side A: track 0, 1, 2 ... 79 , then side B: track 0, 1, 2 ... 79 . Sometimes alternating: side A track 0, Side B track 0, side A track 1, side B track 1, etc. I don't see special flag for it, but it is easy to get order type by looking second track record - if there is 128 for track, then it is alternating type.

A typical protected file looks like this:

   *
     File Header
   *
     Track Header
         o
           Sector Header
         o
           Sector Header
         o
           Sector Header
         o
           …
         o
           Fuzzy sector mask - if exists
         o
           Track data  - may be track image self (public) or image containing sector datas. May missing.
         o
           Sector data x sector count - if not in track image.
   *
     Track Header
         o
           Sector Header
         o
           Sector Header
         o
           Sector Header
         o
           …
         o
           Fuzzy sector mask - if exists
         o
           Track data  - may be track image self (public) or image containing sector datas. May missing.
         o
           Sector data x sector count - if not in track image.
   *
      repeat ...

How it looks in practice ?

How Pasti works in practice ? 
Works well with Steem emulator, and likely with Saint too. HxC support is improved . And we have thousands  of images avaialable around.

But there is a lot of STX images for not copy-protected floppies. Because people can not judge what is really protected, and by some recommendations (do it with Pasti for sure) . Maybe the plan was that some team will verify Pasti images and make list which are really protected, which are OK ? Nothing from it is realised, for sure. Actually, author did nothing, really and literaly nothing in last 4 years. Or maybe 7 years, considering that public imaging tool (v0.4b) is made in 2005. Still beta ! In meantime count of images groved, and poeple want some details. I hope that this will at least partially satisfy needs. I already did SW for converting STX to STT format (only with specific protections, but already have about 60 working STT images, usable with HxC and some even writeable onto floppies). SW for shrinking oversized STX files by dropping unnecessary track images (only for experts). Maybe this can help to add STX support for some other emulator (Hatari)  ? Technically, it seems as great achievement. As flaw I can count lack of checksum for Pasti files. Actually, it would be best to have checksums for all track datas separately. Considering not too reliable ways how STX images are transferred to PC: with transfer floppies, in multiple chunks, or via some cable connection between ST and PC. Even hard disk transfer is not 100 % safe.

The best thing is that is possible to make images of copy protected floppies with stock Atari ST machines. And result of it is really lot of titles imaged. But looking about how author self supports it (or better said, how supports it not) I can only wish that someone other, with more sense for community and communication would do it.

Credits go to Markus Fritze, Klaus Brod and hunderts of people who made Pasti popular by imaging, sending, uploading, hosting image files.

And not to forget Muguk and Dal - 2 admins which banned me for nothing, so I did something useful instead wasting time with useless discussions on forum :-)

For the end, very important issue: reliability of STX images

Are all Pasti images 100% OK ? What are chances that there are errors in STX image ? Usually people test STX images before make them public available. Test in Steem, of course. But how test ? I was not present at any of imaging made by someone other, but can say with confidence that at least 99% tests is made by checking only is game starts, until begin level, or couple levels more. And here is possible PROBLEM : I start with recent case: I imaged game Voyager with public Pasti. Floppy is 22 years old. Only with fourth floppy drive I got working STX image. Running game directly from floppy confirmed that problem is not Pasti - with first drive game worked not on real Atari from original floppy. I examined images, and found that error was exactly in copy protection. It is Copylock, and sector 6 on track 0 was readen incorrect in first 3 imagings. It is slower reading sector with fake CRC error. But must be readen 100 % correct - if not, game will not start. Pasti did not notice error - normal, as there is CRC error reported by FDC. It just confirms that CRC error is put there with purpose, to confuse copy SW. This was straight case, and imaging error was obvious after short testing. But we have a lot of copy protections with delayed effect. or performing check(s) in later game stage(s). It was btw., recommendetation of Rob Northen self too. So, it may happen that protection check fails, but game still will work well until some later stage. Or floppy read for protection check will happen on stage 11, for instance. So, the only reasonable answer is: we can not be in all cases sure that some Pasti image is 100% OK, because there are situations when image may contain error(s), mostly caused by age of floppy disks imaged, + by floppy drive, head align and similar. Pasti HW and SW is not enough sofisticated to detect all possible errors during imaging. So, playing games from STX image(s) until end is required test - about image correctness. It still not proves that every bit is readen correct, but is good check for correct imaging of crucial part: copy protection. Generally, copy protections are what makes floppy imaging very hard and troublesome. And obviously, the place where most likely it may fall. My opinion is that true preservation should include instead images of copy ptotected public releases : the sources of original, and executable SW in not copy protected form. But as things are in real World - it will be not much of such ever. Many source is lost forever, authors, copyright holders care not for preservation. They have no any copies, so for some SW only source is pirate crack. It is big shame for all involved, and maybe just indicates that all what interesting them is money.

And that above is not only some dark scenery, theory: many of Atari ST floppies with game originals after 1989, 1990 have some copy protection with delayed effect. It may be over 200 titles. So, it is almost sure that some STX images are bad, considering that imaging was made when disks were aprox. 18 years old in average.

Update: Expectable problems with Pasti images and what we have online

As said, Pasti imaging is not flawless. Since author does nothing to better inform people about it, not giving any clues about advanced usage, I feel need to write here my experiences and recommendations to make images with less possible errors.

If reading is unreliable some sectors may be readen bad. Then in normal case SW reports CRC errors. Pasti usually warns not, just writes readen, with CRC error flag on - assuming that it may be copy protection. The real bad thing is that quick checking of image will not show any errors, because need to play more-less to reach problematic part. It was case for instance with image of Kid Gloves posted at AF. Average user can hardly determine such problems - but examining STX file may give some clue - if there are CRC errors in middle of disk, over more sectors, it is likely not copy protection.

What average user can, and actually need to do is: taking care about cleanliness of floppy drive. Floppies self is not recommended to clean, but methode of multiple reading will for sure make it cleaner.

There are cases when imaging just don't work, and program stops. I will not go much in that here - reasons may be some not supported protection, or just very bad condition of disk. If cleaning helps not, it is likely too damaged, and impossible to get correct image. Still, it may be worth to keep such floppies. Some parts must be still OK - may help in building image from multiple sources. This initiates thoughts about how to coordinate hunt for missing rare titles ..

Another case, where image can fail is when copy protection is not enough tolerant, and happens with some floppy drives used by imaging. It is certainly rare case, but again, quick test of image may be noot enough. So, it may happen that drive gives index pulse little earlier or later than coders expected, and then copy protection check will detect it as pirate copy. I saw such in case of games To Be On Top and Rubicon. Btw. not enough tolerant code is not so rare case, and not only at copy protections. For instance famous FastCopy Pro 3 had problems with formatting floppies on several floppy drives - for me it never worked. It was corrected in later revisions.

Fortunately, such 'bad' STX images (with little shifted track image) can be used to get 'perfect' copy - by running in Steem Debugger can see fine how copy protection check fails, and correcting STX image by shifting little problematic track image.

Delayed protection problem is likely not only Pasti problem, but general problem with floppy images.

What is good in all this: in fact, testing of STX images is done in many cases. Not in some organised form, but just by playing games until end in emulators. Of course, mostly quality games were played. People could report problems in diverse forums. Normally and especially with Atari ST, you can not be always sure that the problem is bad STX (image), but after some time things getting clarified and some can finish game, and say: it is good  :-)

P. Putnik, STX description Latest update: Jan. 2013.

Back to Data Formats