CBT Tape Home Page

Downloads

CBT
Top 10 Files

Updates
Overflow
JES3

JES2

CBT249 & Older

UCLAmail

Xmit Manager
XMIT Viewers

MVS 3.8J

VM/370 R6

OS/360

OS/VS1

PDS 8.6

Linux/390

MVT Mods

Fish's Herc Tools

PC Tools

AWSTAPE

DSS-FDR Format

AWS Format

ZIP Format Data

Information

Order CDROM
Newsletter

FAQ

Links

Mail & Newsgroups
Fun

Features 
Steve Samson

Contribute files

Freebies

Linking to CBT site
Advertising here
Complaints
Workbench

Cross Links

Contact Us

Buy Gear

cbt-tape on
Yahoo Groups

AWSTAPE Utilities and information

  • What are .AWS nee AWSTAPE files?

An AWS file is a portable virtual tape.  The format was developed and documented,
although not officially, by IBM.  It is used in the IBM P/390 machines, the freeware
Hercules emulator, and most of the commercial emulator products provide support
for AWS format, as well.  An AWS tape is a single binary file, which when
mounted using the correct commands for your emulation platform, appears as a
real tape to the guest operating system.  Tools also exist to import/export real
tapes to virtual, and to directly create an AWS file or extract from an existing
one, without using a real tape drive.  Currently the most robust tools are all
on MVS.
The CBT BASE and OVERFLOW tapes are available to download in AWS format.

 

MVS Tools

File # 477 AWSUTIL to create "virtual tape" disk files

File # 478 RAWSTAPE from Jan Jaeger

File # 533 AWS Tape <---> Real Tape Conversion Utilities on MVS 

File # 585 AWSSL - Dataset to SL AWS Tape Utility - Reed Petty

VM Tools

  • will be added soon

VSE Tools

  • will be added soon

Windows Tools

Rob Story AWS File Browser is a windows tool to browse the contents of an AWSTAPE file

Sam Golob's article on AWSTAPE

                MVS TOOLS AND TRICKS OF THE TRADE
                         DECEMBER 2001
                                      Sam Golob
                                      MVS Systems Programmer
                                      P.O. Box 906
                                      Tallman, New York 10982
Sam Golob is a Senior Systems Programmer.  He also participates
in library tours and book signings with his wife, author Courtney
Taylor.  Sam can be contacted at sbgolob@cbttape.org.  The Online CBT
Tape web site has its own URL, which is www.cbttape.org.
VIRTUAL TAPES
      Today I'm going to talk about a subject that many of us
currently aren't closely connected to.  But I can almost guarantee
that in the foreseeable future, even those of us who never heard of
this subject, will have to deal with it on a daily basis.
      One often does not foresee the future.  You never know when a
"minor development on the outskirts of computing" will make a deep,
lasting impression and will have a permanent effect.  One of these
"minor developments" is the use of "virtual peripheral devices" that
are emulated by disk storage.  For example, if you are running MVS on
a P390 or a system that is similar in concept (such as Hercules), you
have defined "tape drives" and "3380 or 3390 disk packs" that are
really PC or OS/2 disk files.  Someone who has, for years, run MVS on
large mainframe machines, has no exposure to these virtual devices.
However, tape files that have been created as "emulated tapes", and
which are really disk files, will soon be sitting at your doorstep,
knocking on your door.
      Please allow me to mention something else which relates to this
subject.  For years, I have seen misconceptions which people have had
about copying tape data.  Many (otherwise knowledgeable) folks feel
that IEBGENER is about the only tool you need, to copy tape files.
This (in my opinion, erroneous) idea, is based on the concept of a
"tape file" as the principal entity you're handling, and not on the
more realistic idea of handling "the entire tape" as the principal
entity.  A program such as COPYMODS (from File 229 of the CBT Tape
collection of free MVS utilities) copies, and handles, an entire tape,
as though it were ONE object.  It doesn't matter if there are many
hundreds of separate files on that tape--COPYMODS will deal with the
entire tape as a unit, even while copying each file of it.  The newer
versions of COPYMODS (currently at Level 088) can CHANGE the copied
tapes in various ways, such as altering the Volume Label, but in
concept, a program like COPYMODS will handle an entire tape as though
it were one object, even with its new bells and whistles that can alter
the tape label contents.
      Why am I bringing up this apparent digression?  Because it
pictures the idea of an entire tape, with all its files, as ONE object,
and not merely as a COLLECTION of the possibly large number of FILES
that are contained in that tape.
      The idea of a "virtual tape" as a disk file whose scope
encompasses an entire tape from end to end, as ONE object, is about
to burst on the MVS scene.
Current Implementations of Tapes as Disk Files - the VTS
      At this time, many installations are installing Virtual Tape
Systems (VTS boxes).  These are really separate computers, to be
attached to an MVS system, and they act as though they are large
libraries of tape volumes.  The devices employ large amounts of real
memory and disk storage, and they contain backup tape robots, to be used
when the memory and disk storage overflow.  The very idea of a Virtual
Tape System is to simulate tape mounts with disk files.  If a Virtual
Tape System is hooked up to MVS, and a job calls for a tape mount on it,
the "tape" is really a disk file, unless it has been archived.
      The way Virtual Tape Systems are constructed, we, as MVS users,
have no access to the actual file structure of the "tape volume" as a
disk file.  We only care about that VTS "tape" as though it were really
a tape.  However, the P390 "emulated tapes" are an entirely different
story.  It is possible, through an understanding of the file structure
of the disk file which emulates the tape, to be able to transport that
file to other MVS systems, and even to create a "real tape" on a tape
drive, from it.
      The rest of this column will describe the "AWS" disk file format
which represents an entire tape on disk.
The AWS Disk Format of an Entire Tape
      Tapes are by nature sequential.  They contain blocks of data,
which are organized into files.  To represent a tape on disk, you must
know when a file begins, when each block of data begins and ends, and
when each file ends.  Suppose you picture the disk representation of a
tape.  It has to be a sequential file.  But the beginning and end of
the tape files, as well as the blocks of data which comprise them, must
be clearly marked.  In the AWS Tape format on disk, the marking is done
by means of 6-byte headers, that are placed in between the blocks and
files of "tape" data.
      The concept of the AWS Tape header format on disk is very easy
to grasp.  Every header contains the number of bytes in the data block
to follow (it might be zero), the number of bytes in the previous
block (which also might be zero), and an indicator whether this header
marks the end of a file.  If that indicator is not on, then we are at
the beginning, or in the middle, of a file.  If it is on, then we are
at the end of a file.  Tape Labels are themselves only files on a
tape, which "sandwich" the data files.  Therefore, Standard-Labeled
tapes, and Non-Labeled tapes can both be represented in this AWS format.
Conceptually, that's really all there is to it.
      But there is one more small idea which contributes to the picture.
On a PC disk, or as an OS/2 file, the AWS-format "tape" is conceptually
one continuous stream of data, without blocks.  But on an MVS system,
every file has to be logically broken into blocks.  In normal MVS data
management, there's no such thing as a logically continuous stream of
data which is arbitrarily long.  But on a PC or OS/2 disk file, even
though the data is broken up into hardware sectors, the file is
logically one big continuous piece.
      So a small barrier has to be crossed, when you are thinking of
"porting" AWS Tape data on disk, from a PC or OS/2 file to an MVS
system.  You have to arbitrarily break the data into blocks.
      Actually, when you upload any PC data to an MVS system, you have
the same situation.  Usually, the FTP or TSO administrator (when
setting up default IND$FILE uploads) makes a decision as to whether the
default MVS file after an upload, will be FB or VB, and how big the
LRECL and BLKSIZE will be.  You can override their decisions, but some
such decision will have to be made.  The "blocked" nature of MVS data,
forces you to make some decision as to how to re-block any uploaded
data.
      Therefore, an AWS "tape" file on disk, when uploaded to an MVS
system, has to be artificially re-blocked.  Some people (like Brandon
Hill in his AWSUTIL program - File 477 of the CBT Tape) have decided
to block the data with RECFM=VB.  I have decided, with my new set of
AWS utilities (File 533 of the CBT Tape) to block the data as card-image
files:  RECFM=FB and LRECL=80, with any convenient blocksize.  If you
FTP the MVS data back down to a PC file, you again can get a usable
and "continuous" PC file.  This MVS re-blocking only "means something"
if the data is to be further manipulated on an MVS system.
Specifics of the AWS Tape format
      Now we'll talk about the specific structure of the AWS headers,
which delimit and describe the tape "blocks" and "files" in the AWS disk
representation of a tape.  Each six-byte AWS header is divided into 3
two-byte sections.  And the first six-bytes of any AWS "tape" file, is
always the first header, which contains enough information to find all
the rest of the AWS headers in the file.
      The first two-byte section of the AWS header contains the number
of bytes to be found in the next block of tape data.  Since this is a
two-byte field, the number contained in it, can be as large as 65535,
or 64K bytes.  But there is one hitch.  The number is not a normal
halfword quantity.  Instead, it is backwards, in the famous "Little-
Endian Format".
      I'll explain.  Suppose we want this field to represent the number
32720, to tell us that the next block of "tape data" will contain 32720
bytes.  The hexadecimal representation of 32720 is X'7FD0'.  But the
first halfword in the AWS header will not contain X'7FD0'.  Instead, the
two bytes will be backwards, and the field will contain X'D07F'.  This
is confusing to us MVS'ers who have never dealt with Intel or Microsoft
programming.  Therefore, any MVS-based program which deals with the
numeric quantity contained in this two-byte header field, must first
reverse the two bytes.  Little-endian is ubiquitous here.
      The second two-byte field in the AWS header, contains the number
of bytes of tape data contained in the previous "block".  Of course, if
our header is at the beginning of a file, there are no previous blocks
in this file, and the second field will contain X'0000'.  Remember that
this second two-byte field is also in little-endian format, and any MVS
program dealing with the quantity, will have to reverse the two
bytes.
      The third two-byte field is the simplest of all, in the current
implementation of AWS tape format.  The last 3 hex numbers have to be
zeros, and the first number is either a X'A' or a X'4'.  A X'A' means
that this header does not mark the end of a file.  A X'4' means that
is does mark the end of a file.  Officially, in the description of the
AWS tape format, there's a bit more detail, to cover data chunks that
are smaller than the size of the tape blocks.  But practically, that's
all there is to it!
So How Does It Work?
      Let's look at the first few headers of a Standard-Labeled (SL) AWS
tape.  We'll start with the representation of the first header labels at
the beginning of the tape, including the VOL1 label, the HDR1 label, and
the HDR2 label, each of which is an 80-byte record.  These three 80-byte
records constitute the first actual "file" of an SL tape.
      The sequence of AWS header records describing this first label
file is:
      X'50000000A000' , then X'50005000A000' , then X'50005000A000' ,
then X'000050004000' .
      Let's interpret:  The first field of the first header contains
X'5000'.  This is little-endian format, so the bytes have to be
reversed, to make X'0050', which is decimal 80.  That tells us that the
following block (really the VOL1 label) will contain 80 records.  The
second field of the first header contains X'0000', which reversed, is
still X'0000', so we know that no file data has preceded this header.
The third field is X'A000', so we know we're not at the end of the file
yet.  After advancing 80 bytes past the end of the first header, we come
to the second header.
      The second header (really before the HDR1 label) contains the
quantity X'50005000A000'.  First field X'5000', when reversed, gives
X'0050', telling us that another 80 byte block of data will follow this
header.  The second field X'5000', which when reversed is X'0050', lets
us know that the previous block of data contained 80 bytes also.
Finally, the X'A000' in the third field, tells us that we're not at
end-of-file yet.   The first field has told us to advance 80 bytes past
the end of this header, to come to the next header.
      The third header in our sequence of four, contains the same
quantity as the second header, namely X'50005000A000'.  This tells us
that the block which follows (the HDR2 label) will contain 80 bytes,
the previous block (the HDR1 label) contained 80 bytes too, and the
X'A000' in the third field tells us that we are still not at
end-of-file.
      Finally, after advancing another 80 bytes, which the first field
in the third header has ordered us to do, we come to the fourth header.
This fourth header contains X'000050004000'.  Its first field, X'0000',
when reversed, still X'0000', tells us that zero bytes of data will
follow this header.  The second field, X'5000', which reversed, is
X'0050' or decimal 80, tells us that an 80-byte block has preceded this
header.  Finally, the X'4000' in the third field, informs us that we
have reached the end of this file.  Since the first field of this
header contained X'0000', we expect another header to immediately follow
this header, in the AWS tape file.
Some Practical Correspondences
      Now that we've seen an entire file in AWS format, we can figure
out what the headers signify to us, in conventional tape terms.  Let's
first consider that we're writing this tape, which the AWS headers are
describing, to a real tape drive.
      First, whenever there's a X'4000' in the third field of a header,
that means that we write out the last block of tape data (on a real
tape) and then we write a tape mark.  If the third field is X'A000', but
the second field is not zeros, then we write out the number of bytes in
the second field, without writing a tape mark.  Finally, if the second
field is zeros, and the first field is not, we advance the tape by the
(reversed) quantity in the first field, reading in that number of bytes
of data.  No writing is done yet.
      Now, let's imagine that we're reading this tape.  The first header
tells us that we must look into the first field.  If that is not zero,
then we reverse the two bytes, and read forward, by the number of bytes
thus described, coming to the second header.  If the third field in the
second header is X'4000', we are at end-of-file.  Otherwise, we look at
the second field in the second header.  That should correspond with the
number of bytes we just read.  Finally the first field in the second
header tells us, that we must advance that number of additional bytes of
data, to get to the third header, and so on.
      I hope that this month's topic has opened your eyes to the
beginning of a new era in "tape" processing.  Files describing an
entire tape, can now be put on disk.  On an MVS system, whether these
files be "folded over" in FB-80 format (as in CBT File 533), or VB
format (as in CBT File 477), the data can still be dealt with, as
tape data.  I have written a program to convert the FB-80 format data
into a real tape on an MVS system (see File 533 - program VTT2TAPE).
I sincerely hope that you will all expand your minds to include these
new concepts in your thinking.  Best of everything!  See you next
month.

 

 

 

 

Copyright © 2000,2002 cbttape.org   All rights reserved.
All trademarks are properties of their respective owners.