17 July 2000
Since May 1999, my column here in Byte.com has been titled "Utility Infielder". The name, taken from baseball's term for somebody who can play most positions, gave me wide latitude (and longitude) to write about subjects that didn't fit into other columns in Byte.com. Regular readers of "Utility Infielder" at Byte.com know I've written about OS/2, and some on other operating systems and OS-level issues. My column has also included everything from protecting aginst viruses and Year 2000 to "why is it hard to improve programmer productivity?" It was a challenge, and fun too, always something new coming around the corner. I enjoyed it, but of late had wondered about writing to a more specific focus.
By sheer coincidence (really), Byte.com's Executive Editor called to discuss a potential change in focus. A lengthy discussion led to a general agreement that I would change focus to delve into advanced areas of computer technology while continuing to cover operating systems.
The new column title we came up with to reflect this refocus, effective as of this month, is Advanced Software and Technology. I'll be covering a range of operating systems, including OS/2, the whole BSD family, a number of unusual ones like Inferno and Plan Nine, SCO and other unix variants like AIX and Monteray. Since Windows, Linux, Mac and BeOS are well covered elsewhere in Byte.com, but that still leaves me more than a dozen OSs to cover.
In addition to operating systems, I'll be looking at what is being done at the bleeding edge of technology, both hardware and software. A major part of this will be exploring that edge and writing about the interesting systems and challenging applications. These new technologies will indirectly and directly affect us with more accurate weather prediction, greater bandwidth, better graphics, solutions to pollution problems, and a host of other discoveries. I'll be starting that journey of discovery with one of my favorite sites - the San Diego Supercomputer Center.
My Advanced Software and Technology (ASAT) column is aimed at exploring the outer and upper reaches of the computing industry, with attention on how it will affect all of us. I will write about new technologies that may someday reach your system, alternate operating systems with their special features, and supercomputer systems and applications.
ASAT will vary in content each month, but the goal is to have two to four sections in each column. They will be:
Advanced OS Topics
Each section will build on earlier sections, with links to refer to relevant earlier sections.
In writing about Advanced OS Topics, I will cover clusters, partitions and large system optimization techniques. Clustering and partitioning are ways to configure systems for a specific need. Clustering ties multiple independent systems into one functional system with fail-soft or fail-safe characteristics. Partitioning is a method of taking one big system and making it behave like several smaller systems, logically independent but sharing common physical hardware such as power and control. Large systems, those with many processors and more than 4 GB of memory, have special configuration needs. These and other advanced OS subjects will be one part of a typical ASAT column.
Supercomputer Systems, called supers for short, will look into high end computing to see what it takes to make a super, and what neat and interesting things are being done with them. One of the challenges with supers is solving the hardware and software problems that arise when pushing beyond the leading edge of technology. I'll report on important advances when they are made in supers, and whether they might filter down to your computer.
Super Applications will discover what the small and large supers are doing and how it affects us as individuals or as a country. Some of these applications are freely available for small scale experiments, and some only function on supers.
Interesting Technologies will cover new technologies more than a year or two away from your desktop, plus software and applications that are off the beaten path. This section is also a place for those items that catch my interest but don't fit anywhere else.
As always, I'm glad to hear from the readers with questions, suggestions and even the occasional correction. Feedback is welcome and will help guide me in searching out new things. Write me at infield@yelmtel.com.
SDSC is the San Diego Supercomputer Center. SDSC is a research unit of the University of California and the leading edge site for NPACI, the National Partnership for Advanced Computational Infrastructure NPACI.
There is a danger in exploring SDSC. It is easy to get sidetracked into areas you hadn't known about, and it can be a big timesink. Writing this column was difficult because I wanted to go off and just explore, but I had to pull back and put together the tour. One of the fun things is to click on whatever image shows up in the image jukebox. This will take you to a larger image and an explanation, with further links of course. All in all, I'd suggest setting an alarm before you start exploring this site so you'll get back in time for dinner.
The SDSC web site covers a broad array of disciplines:
Each of these disciplines has a page listing related projects and services. One neat link is via the Earth & Space section to a 15 year collection of earth photos shot by the astronauts. You can select from a form search, political subdivisions or topographic areas: Earth Photos. The NASA Mars missions are also linked there.
Come with me as I explore one of tomorrow's big developments - Digital Libraries, a part of the computing discipline. Digital libraries are the bookcases of the future. Inside your PC will be a link to libraries bigger than the Library of Congress, covering any subject you can imagine, right at hand. Text, pictures, raw data, all are part of the library. As big as the list at SDSC is, the future digital libraries will be immensely larger.
Collecting libraries under one roof is no longer necessary. A new technique called Federation< is being used to handle the collections of libraries that are distributed around the world. This technique creates a distributed directory, the electronic equivalent of a huge library card file, which links to the individual libraries and provides a standard user interface as well as masking the differences among a wide variety of actual data sources. Federation will be used for the Digital Sky Project, discussed below, to bring together the massive collections of source data, plate, film and digital, that astronomers have collected over the past few hundred years.
The Digital Library list below is typical of the links found at the SDSC site. Keep in mind that these 20 links represent only one subject of fourteen under the Computing discipline, which is just one of eleven disciplines. All that, plus links to thirteen scientific organizations, makes SDSC a treasure-house of information.
As an example, let's look at the Digital Sky Project (DSP) at CIT from the list of digital repositories and portals: Digital Sky Project. The DSP is planning to bring together, in an organizational sense, billions of sources of astronomical data into one distributed system. The resulting system will enable access by many people with one interface to all astronomical data, regardless of form or location. The 2MASS infrared survey alone, one of nine listed, is expected to deliver 4,000 Terabytes of digital data. (4 petabytes or 4 x 10**15 bytes or 4 billion megabytes). The size of the DSP project is huge, but it will enable new discoveries and better understanding of the universe we live in. Links from DSP will take you to specific surveys and dozens of other astronomy sites.
SDSC operates the world's largest High-Performance Storage System (HPSS) -- already at 160,000 gigabytes (160 terabytes) and growing: HPSS Statistics. Statistics on one hour of use on July 31, 2000 were impressive:
A typical day moves 250 GB of data to/from the computer systems. Total space used is now up to 182 Terabytes, of which only 1.5 TB are disks, the rest is stored on tape. This is currently the largest HPSS, but storage for the 2MASS survey will have to be almost 22 times as large. Your home system with a 30 GB disk would fill up in 27 minutes at this hourly rate.
More information on HPSS can be found at the worldwide support site: HPSS Support. HPSS is the result of a collaboration of many sites around the world starting in 1993. As a cooperative development between government and industry nearing its fifth release, it has been a substantial success. Both IBM and Sun offer HPSS support on their systems as a commercial product. Near me, the University of Washington uses an HPSS for backup of the campus wide servers UW HPSS.
For more examples of HPSS, check out HPSS in Action and click on the "HPSS in Action" button.
Our last stop on this tour is at the site of the IBM Teraflop machine, the Blue Horizon. Installed earlier this year, it began production on February 9. (A Teraflop is a Trillion Floating Point Operations per second, thousands of times faster than the fastest current personal computer.) Blue Horizon includes 1152 Power3 processors from IBM, each capable of 888 Megaflops. The 576 GB of RAM memory is distributed on 144 eight processor systems and 12 two processor systems.
This monster resides in 42 towers covering 1500 square feet of floor. It is as large as a classic 70s mainframe, the IBM 3033 of the 370 series. For roughly the same space and power, the 3033 generated 3+ MIPS, amazing in those days. Today your PC runs on 200 watts and computes about a hundred times as fast while sitting on a corner of your desk. That's called progress.
Before checking out of SDSC, keep up with new developments by visiting the subscription site and sign up for email to keep informed of what SDSC and NPACI are doing. Subscriptions have three options: Subscriptions. Press releases when they happen, a bi-weekly newsletter or the quarterly Envision publication.
I hope you have enjoyed this peek at one of America's biggest supercomputer sites. This tour has barely scratched the surface, perhaps one-tenth of one percent of the information available at SDSC without counting other linked sites. It's a fascinating scientific world out there on the net.