Article 159442 of comp.os.vms: In article <56l9fm$6em@nntp.ucs.ubc.ca>, shoppa@alph02.triumf.ca (Tim Shoppa) writes: > In article , > Terry C. Shannon wrote: >>shoppa@alph02.triumf.ca (Tim Shoppa) wrote: >>>In article , >>>Terry C. Shannon wrote: >>>>In a nutshell, Galaxies is a software architecture that will enable >>>>clusters in a cabinet. Instead of building a box with, say, 32 CPUS (and >>>>gee, I wonder who might be planning to do just that???) and running 'em as >>>>a mongo SMP system, a Galaxies box might contain 32 CPUs partitioned into >>>>8 4-way SMP nodes which are clustered together and which have node-private >>>>memory and I/O as well as access to application and system shared memory. >>> >>>Does this mean that you'd have to pay DEC for 8 VMS licenses too? >>> >>>Tim. (shoppa@triumf.ca) >> >>Now that's a good question. I kinda doubt it, but you never know... > > We were thinking about buying "boxes" from DEC a few years ago > that would have been 4 4-processor SMP systems, all mounted > in a single rack cabinet and clustered. Running such a box would, > indeed, have required 4 VMS licenses (or in our case, paying a CSLUG > fee 4 times larger than if it was a single "system".) I was at DECUS...but I missed Steve Z.'s presentation. :-( OTOH, I talked extensively with Fred Kleinsorge, Steve Z., and Burns Fisher at various times afterwards. Fred and Burns, and others from VMS engineering, were extremely excited about this idea (as was Steve, but honestly, I only asked him for copies of his transparencies :-} ). Note that Steve's presentation will be on the DECUS web site, along with the other presentations from Anaheim, although I don't know when that will happen. My take is the following (surmised from my conversations with VMS engineers). There is a basic scaling problem with SMP systems in that they are not _totally_ symmetric. Under VMS, the primary CPU handles all the timer interrupts, the I/O _completion_ interrupts, and one other item I can't recall. As a result, and depending on your work load, there is a "knee" in the performance versus number of cpu's curve such that adding additional cpu's gives proportionately _less_ additional performance per cpu added. In VMS, this occurs at around 4 cpu's for a "typical" general work load (judging from the VUP's marks for SMP VAXen). Other operating systems may have the knee at a different number of cpu's, but they certainly have a knee. So the basic problem is a lack of scaling when going to large numbers of CPU's in an SMP configuration, where large is, say, 32. OTOH, VMSclusters scale with number of nodes very well. Now take note that in VMS 7.x, there are plans to add cluster-wide global sections (and make other system service kinds of things cluster-aware). Then comes the "Ah ha!": you can overcome the lack of SMP scaling by grouping CPU's into nodes (within a single system) and cluster the nodes. As was said already, VMS clustering is known to scale very well. The benefits come in several flavors. First, inter-node communications are done at _system bus_ speeds, not CI or FDDI, etc., speeds. Think of how quickly and easily locks and lock trees could be remastered. Secondly, while each "virtual node" would have its own copy of VMS running in (a partition of) memory, there is also the possibility of direct shared memory, a la the VAX 782. Those cluster-wide global sections now become a single copy in shared memory accessible at memory speeds. (Perhaps the lock trees go in shared memory as well?) Third, presumably each virtual node can handle that node's I/O completions, etc., so that many nodes in the _system_ can process I/O completions, instead of just one, thus greatly increasing the total I/O a system is capable of. This last would take some redesign of, or help from, the hardware I expect. Fourthly, (I think Terry mentioned this) you could _dynamically_ move cpu's from one virtual node to another according to the current load conditions. For example, a node that was supporting a lot of compute intensive jobs could benefit from a larger number of cpu's than one that was primarily doing I/O. As the load changed, the number of cpu's could be changed to accommodate the load. There are some other goodies that come with the Galaxy design in mind, such as hot-swappable cpu's and memory modules. As for Tim's question about licensing, my _guess_ is that this would be treated as a single system and licensed as such (perhaps requiring a VMScluster license in addition to base VMS and an SMP VMS upgrade). The point is to turn a non-scalable, SMP _system_ into a scalable one, not to just repackage many systems into one box. The selling point will be the scalability of a 32-node system, and the performance gains that implies. As far as running multiple operating systems on a single system, as some else asked, the answer is a resounding NO! This is a VMS initiative to have VMSclusters "raise the bar" in clustering technology far above its current level, thus keeping VMScluster technology far ahead of the "Johnny come lately" clusters that other operating systems offer, or will offer in the near future. [However, given the hardware support, it's not unreasonable to think someone might try to "hack" a multiple operating systems onto the single hardware system...] One last note: Galaxy is just an "idea" today. No money has been allocated for development and no feasibility studies have been done. Nevertheless, almost everyone who has heard the idea, inside and outside of Digital, seems to be quite excited about it. I know I am! -Ken -- Kenneth H. Fairfield | Internet: Fairfield@Slac.Stanford.Edu SLAC, P.O.Box 4349, MS 46 | DECnet: 45537::FAIRFIELD (45537=SLACVX) Stanford, CA 94309 | Voice: 415-926-2924 FAX: 415-926-3515 ------------------------------------------------------------------------- These opinions are mine, not SLAC's, Stanford's, nor the DOE's...