M.P.A. -- Memory Performance Analyzer ---------------------------------------- (C) Copyright 1988 by Rand P. Hall and M.P.Gerlek All Rights Reserved 21-Mar-88 : v1.0 (initial test) Designed by: Rand P. Hall Coded by: M.P.Gerlek 910 Riverside Drive 2 Robinson Drive Methuen, MA 01844 USA Bedford, MA 01730 USA "All lovers swear more performance than they are able, And yet reserve an ability that they never perform." - William Shakespeare, "Troilus and Cressida" (III,ii,89) Program Outline --------------- 1. Start foreign command invokes Main.For command line is parsed out, cli$dispatch calls requested routine A. SVASS (type 1 rules) start_svass - open log file peek out values test against rules, output texts to log file stop_svass - close log file B. MPAP (type 2 rule collection) start_mpap - open log and data file calc start/stop times hibernate until start time while (now < stop_time) do hiber for interval get_rates get misc values get_process_data get_rates again calc delta-rates set_switches - determine findings_bitfield if possib violation, store bitfield enddo store collected per process data stop_mpap - record cycles, close log and data file C. ANLZ (type 2 rule analyzation) start_anlz - open log and data file, get count threshhold store rule_array for each finding stored with every every elem of rule_array if (rule and finding)=rule then this rule is broken output_texts - if broken enough, output to log Sources Contained Within ------------------------ AAAREADME.TXT - this documentation ANLZ.FOR - sources for mpa/analyze DEBUG.MMS - mms description file for debuggable version DESCRIP.MMS - mms description file for production version MAIN.FOR - sources for calling main of mpa MPA.OPT - options file for linker MPAP.FOR - sources for mpa/collect MPA_CLD.CLD - command definition file QUADMATH.MAR - support for time calculations SAMPLE.MAR - support for mpa/collect SETUP.COM - command procedure to facilitate use SVASS.FOR - sources for mpa/brief [.BATCH]COLLECT.COM - command procedure to collect & analyze in batch [.BATCH]SUBMITTER.COM - command procedure to submit above to batch Rules Condition / Data File Coding ---------------------------------- \conditions \ rules\ 15 14 13 12 11 10 9 8 7 6 5 4 3 2 1 0 Value in Hex \ -- -- -- -- -- -- - - - - - - - - - - ------------ normal: X X X X X 060B rule 1: X 0001 rule 2: X X 0028 rule 3: X X X 0052 rule 4: X 0400 rule 5: X X X X 0B02 rule 6: X X X 00A2 rule 7: X X X X 1320 rule 8: X X X X 2B00 rule 9: X X X X X 4344 rule 10: X X X X X X C344 condition 0: system pagefault rate > 0.5 condition 1: system wide pagefault rate > 100 condition 2: system wide pagefault rate < 75 condition 3: wsx too small condition 4: mem for a proc < 80% wsquota condition 5: free memory available condition 6: no free memory available condition 7: procs w/ loans used but not at wsextent condition 8: inswap rate > 1 condition 9: cpu idle > 10% condition 10: hardfaults > 10% of system-wide fault rate condition 11: balance set slots available condition 12: no balance set slots available condition 13: noswap process exist condition 14: noswap procs = 0 condition 15: no procs have loans MPA - Directions --- ---------- How MPA works. MPA is designed to assist you in maximizing the performance of your memory subsystem. MPA performs both the collection and analyzation parts of the tuning process. MPA has an internally defined set of rules. It collects the data necessary to analyze the rules and then after analyzing them it explains the problems it found and suggests ways to eliminate the problem. MPA has two different sets of rules. The first set analyzes the composition of the portion of memory occupied by the operating system. If too large this portion of memory will take away memory that could be used by user processes. If too small the operating system could start paging itself out of memory degrading the performance of all processes. MPA attempts to optimize the size of system-occupied memory with its type 1 (aka "SVASS", system virtual address space sizer) rules. We have found that Autogen typically makes non- paged pool much too large. Rules of type 1 correct this. We've witnessedd savings of .25 - .5MB on the systems we've tested this on. MPA's type 2 ("MPAP") rules attempt to optimize the way the operating system manages its memory. VMS' memory management system can be tailored to react in certain ways to certain situations, hence it can be altered to react favorably to your particular work load. MPA analyzes your work load and the way memory management currently functions and suggests ways to tailor memory management to your system's environment for maximum output. How to use MPA. MPA is simple to use. To collect and analyze data of straight-forward type 1 rules, issue the command: $ MPA /Brief MPA will poll the components of the operating system that make up its memory resident portion. If it finds that any part of this memory is configured in a detrimental way it will tell you the problem and suggest remedial action. As the qualifier /Brief suggests, this analysis of type 1 rules takes only a few moments and may be done interactively. A log file will be created containing the actual texts; the log file defaults to the name MPA.LOG, but may be changed with the /Log qualifer, for example: $ MPA /Brief /Log=tuesday.dat Collecting and analyzing data of type 2 rules is different. MPA collects the data over a suitably representative period of time and stores it for later analyzation. This allows MPA to see a representative sample of your workload. To collect type 2 data issue the command: $ MPA /Collect Once MPA has finished collecting its data issue the command: $ MPA /Analyze This analysis will produce problems and solutions just like MPA/Brief. The time needed to complete this analysis will vary depending on how much data was collected, but in general should be very short. MPA is designed to minimize the amount of data it stores, trying only to store data that may be indicative of problems. Therefore processing time is likely to be proportional to the number of problems MPA finds. Furthermore, the /Collect option has several optional qualifiers. The /Output qualifier directs the data collected to a different file than the default, MPA_Collect.Dat. The /Start and /Stop qualifiers specify when and for how long MPA is to run. The /Start qualifier uses an "absolute time" argument (default is "now", "-- ::.") and the /Stop qualifier uses a "delta time" argument representing the amount of time from Start for which data is to be collected (default is one day, "1 00:00:00"). The /Interval qualifier is aslo a delta time argument and specifes how often MPA is to collect data (default is one minute, "0 00:01:00"). To collect data for two days in half-hour units, starting at midnight on December 4th, the command would be: $ MPA /Collect /Start = "4-DEC-1988 00:00" - /Stop = "2 00:00" - /Interval = "0 00:30" The /Analyze routine has an /Input qualifier, if you wish to analyze a file other than MPA_Collect.Dat. If you have changed the /Start, /Stop, and /Interval qualifiers, you may also wish to change the /Count qualifier in the Analyze step. The count determines how many violations of one rule indicate a truly broken rule (i.e. just one five minute occurence of bad behavior in a 24 hour period is not necessarily a bad thing). For the default collection period with one minute intervals, the default value of /Count is 15. Again, log files are created by this portion of the program. Both the /Collect and /Analyze sections create log files called MPA.LOG, but these may be changed with the /Log qualifier. Suggested Method of Use. It is suggested that you use MPA/Brief first for a few reasons. First, it is possible to see DRAMATIC increases in memory performance very quickly. Second, if you don't use MPA/Brief first MPA/Analyze will pick up problems that may be caused by or influenced by things that could be taken care of with MPA/Brief. It is also suggested that you run MPA/Brief after you system has experienced one peak of activity since the system was booted. The operating system keeps some counters and table sizes that only become informative after the system has experienced its heaviest typical load. After running MPA/Brief and acting on its suggestions you should run MPA/Collect. As this process takes, by definition, a sizable amount of time to complete you may wish to run this as a batch job or detached process. Since it does take so long MPA/Analyze is not coupled with MPA/Collect. You can run MPA/Analyze anytime after MPA/Collect has finished. Command Summary --------------- $ MPA :== $dev:[dir]mpa_system ! foreign command Syntax: MPA [/log] /brief /collect [/output] [/start] [/stop] [/interval] /analyze [/input] [/count] ! default data storage file is mpa_collect.dat ! default log file is mpa.log ! default start is "now" ! default stop is 1 day (24 hrs) ! default interval is 1 minute ! default count is 15 Command Setup / Example Usage ----------------------------- *** * Important: site-specific changes must be made in two * command procedures included in this release, * SETUP.COM and COLLECT.COM. These changes * (logical names mpa$main and mpa$batch) must be * made before using MPA as suggested below. *** To run a basic test of the MPA system: 1. $ @setup This command procedure will define the needed commands and logicals to build and run MPA. 2. $ MMS /Log This will invoke the MMS utility to build MPA - MPA_System.Exe - using the default file Descrip.Mms. (To build a "debug" version of MPA - MPA_Debug.Exe - the alternate command file Debug.Mms may be used. Note that the debug version cannot be used in the steps below. 3. $ MPA /Brief /Log=Brief.Log This command will run MPA using the type 1 ("SVASS") rules. Examine the log file for MPA's recommendations for system improvement. 4. $ Collect This command will run, in batch, a type 2 MPA collection and analyzation. A command procedure (Submitter.Com) is invoked which will prompt you for the needed start, stop, interval, and count values as well as an ID string for the run (the defaults are recommended for an initial test). This data will be passed to another procedure (Collect.Com) which is submitted to sys$batch: this procedure runs MPA/Collect, MPA/Analyze, and then mails the log file to the user. (N.B.: MP/Collect requires CMKRNL and WORLD privileges.)