F R A G M E N T A T I O N I T S C O S T S , C A U S E S , A N D C U R E S John Claxton Collier-Jackson, Inc. (813) 872 - 9990 x367 CompuServe 70003,4140 FRAGMENTATION o What is it? o What are its costs? o What are its causes? o How do you find it? o How do you fix it? FRAGMENTATION Goals o To establish the cost of fragmentation o To determine possible causes of fragementation o To discuss possible solutions to fragementation FRAGMENTATION Non Goals o A detailed discussion of Files-11 I/O Subsystem o A detailed discussion of diagnostic software o A detailed discussion of third party solutions FRAGMENTATION A Definition The condition of logically contiguous disk space being physically noncontiguous. FRAGMENTATION Disk Free Space A disk's free space if fragmented when the unallocated blocks on that disk are noncontiguous. FRAGMENTATION File Extents A file is fragmented when the file's extents are noncontiguous. FRAGMENTATION Disk Free Space The fragmentation of a disk's free space may be thought of as potential file fragmentation. FRAGMENTATION Of A File The fragmentation of a file's extents may be thought of as actual fragmentation. FRAGMENTATION Window Turns A window turn occurs when an I/O request occurs for a file with multiple retreival pointers which are not currently in that file's window control block (WCB). FRAGMENTATION FRAGMENTATION Split I/Os A split I/O transfer occurs when a single I/O request spans multiple extents of a file. FRAGMENTATION Its Costs o Increased I/O Times (Split I/Os) o Increased Memory Consumption (Larger WCBs & Buffers) o Increased CPU Overhead (Extended QIO Processor) FRAGMENTATION I/O Times o I/O Software Processing 8% o Controller Latency 2% o Seek Time 58% o Rotational Latency 20% o Transfer Time 12% FRAGMENTATION Costs: An Example Q: How much does fragmentation costs? Method: Determine the time spent doing split I/Os, Place a dollar value on system time, And mulitply. FRAGMENTATION (An Example Cont.) Avg. Split I/Os Per Second: 10.0 Overhead per Split I/O: 30 ms System Time Per Hour: $30.00 FRAGMENTATION (An Example Cont.) (10.0 I/Os per Second) * (3600 Seconds per Hour) = (36000 I/Os per Hour) (36000 I/Os per Hour) * (30 ms per I/)) * (Hour per 3600000 ms) = 30% FRAGMENTATION (An Example Cont.) ($30 per Hour) * (8 Hour per Day) * (5 Day per Week) * (50 Week per Year) = $60,200 30% of $60,200 = $18,060 (Approximate Cost of RA82) FRAGMENTATION It Causes o Dynamic File Extension o System Utilization: Digital o System Utilization: Users FRAGMENTATION Dynamic File Extension During file I/O, VAX/VMS will automatically allocate free blocks to a file on an as-needed basis. FRAGMENTATION Courtesy of Digital o Check Compiler's Defaults o ACCOUNTNG.DAT, etc. o RMS File Extension FRAGMENTATION ACCOUNTNG.DAT o $ SET ACCOUNTING/DISABLE o $ ANALYZE/RMS_FILE/FDL ACCOUNTNG.DAT o $ EDIT/ ACCOUNTNG.FDL o Make BEST_TRY_CONTIGOUS = yes o Maybe Adjust EXTENSION Value o $ CONVERT/CREATE/FDL=ACCOUNTNG.FDL - ACCOUNTNG.DAT ACCOUNTNG.DAT o $ SET ACCOUNTING/ENABLE FRAGMENTATION ACCOUNTNG.DAT - BEFORE o BEST_TRY_CONTIGUOUS = no o Total File Extents = 2,035 o Total INDEXF.SYS Records = 21 o Run-time Accounting Information: CPU Time: 4:19.74 Direct I/O Count: 4,179 FRAGMENTATION ACCOUNTNG.DAT - AFTER o BEST_TRY_CONTIGUOUS = yes o Total File Extents = 43 o Total INDEXF.SYS Records = 1 o Run-time Accounting Information: CPU Time: 3:19.07 Direct I/O Count: 2,013 FRAGMENTATION Courtesy Of Users o File creation & extension o Lots of little files o Lack of housekeeping FRAGMENTATION How Do You Find It? o FILEFRAG.COM o DISKFRAG.COM o SHOWSPLT.COM o SHOWWNDW.COM FRAGMENTATION FILEFRAG.COM o Accept file name(s) o Dump file header(s) o Parse dumped headers for retreival pointers o Display fragmentation information FRAGMENTATION (c) 1988 FILE FRAGMENTATION 5-OCT-1988 CJI Inc. 10:21:31.77 FILENAME FRAGMENT ATTRIBUTE PROLOG CTSTACCT.DAT 31 CBT 3 CTSTKSAM.DAT 5 CBT 3 DISKFRAG.COM 8 NOTCBT 0 FILEFRAG.COM 1 NOTCBT 0 SHOWSPLT.COM 1 NOTCBT 0 SHOWWNDW.COM 2 NOTCBT 0 FRAGMENTATION DISKFRAG.COM o Determine the free block count o Create file to fill free space o Dump file's header & parse dump for retreival pointers o Display fragmentation information FRAGMENTATION Disk: D9 Date/time: 14-OCT-1988 15:23:35.09 MiniValu MaxiValu TotBlcks Percntge FrgmtCnt LoBlkCnt HiBlkCnt ======== ======== ======== ======== ======== ======== ======== 1 49 1240 00.19 151 2 48 50 99 1666 00.26 21 50 98 100 149 1094 00.17 9 100 134 150 199 1198 00.18 7 154 186 200 249 868 00.13 4 202 232 250 500 6524 01.03 17 252 496 501 749 8230 01.30 13 512 728 750 999 9636 01.52 11 750 998 1000 1249 7384 01.16 7 1000 1186 1250 1499 2554 00.40 2 1272 1282 1500 1749 3320 00.52 2 1646 1674 1750 1999 7464 01.18 4 1760 1984 2000 2499 11094 01.75 5 2104 2400 2500 2999 13438 02.12 5 2514 2886 3000 3999 27772 04.39 8 3018 3920 4000 4999 22916 03.62 5 4232 4932 5000 7499 53276 08.43 9 5078 6948 7500 9999 34754 05.50 4 7996 9148 10000 12499 21066 03.33 2 10522 10544 12500 14999 54962 08.69 4 13082 14994 15000 17499 32268 05.10 2 16054 16214 17500 19999 19690 03.11 1 19690 19690 20000 24999 70862 11.21 3 21662 24798 25000 49999 94406 14.94 3 25526 42738 100000 249999 124138 19.64 1 124138 124138 ======== ======== 631820 300 FRAGMENTATION SHOWSPLT.COM o Reads the PMS database for PMS$GL_SPLITS o Displays current split I/Os, split I/Os since boot, average split I/Os, ... FRAGMENTATION SHOWWNDW.COM o Reads the PMS database for PMS$GL_TURNS (V4.x) EW_PMS$GL_TURN (V5.x) o Displays window turn rate, window turns since boot, average window turns, ... FRAGMENTATION How To Find It o DECUS supplied routines o Several third party tools o $MONITOR IO , FCP o Homebrew software FRAGMENTATION How Do You Fix It? o Digital's Method o Third Party Alternatives FRAGMENTATION Digital's Solution o BACKUP / IMAGE o BACKUP / IMAGE o Elapsed Time (RA81 - TA81 - RA81) approximately 2 hours FRAGMENTATION Third Party Alternatives o Other DECUS Sessions o Trade Shows o Trade Periodicals FRAGMENTATION ????????????? ? ? ? QUESTIONS ? ? ? ?????????????