Today's competitive corporate environment requires nimble IT departments with the capability to respond quickly to changes in capacity and the use of innovative methods to reduce time to market for new applications and systems. Escalating costs for power, raised floor capacity and administrative requirements also drive the need to utilize technology in new ways to maximize a company's IT investment.

Virtualization is an excellent vehicle to address business needs while controlling costs. The IBM* System p* Advanced Power Virtualization (APV) offers advanced technology to facilitate server consolidation, reduce costs, increase utilization and adapt capacity to quickly meet demand. APV, standard on the p5* 590 and 595 servers, can be used to reduce the need for static adapters and respond to increasing capacity demands.

Increasing Utilization

A common implementation benefit of server consolidation is curing the malady of underutilized servers. Most companies without virtualization and server consolidation report UNIX* utilization under 25 percent, indicating underutilized servers.

APV facilitates server consolidation by allowing rapid response to changes in memory or CPU as well as removing the need to physically move I/O or network adapters. System p customers typically drive 60- to 80-percent utilization using virtual CPUs, Micropartitioning* and capped/uncapped capacity techniques. Capacity Upgrade On Demand is designed to provide rapid deployment of spare CPUs. The unparalleled granularity for CPU virtualization that System p virtualization may provide places APV in a category of its own in increasing flexibility and driving better utilization of IT resources.

Cost savings can be realized not only in reducing the number of servers and network and I/O adapters, but also in reducing floor space, power and cooling. Some companies see significant reductions in their overall capacity spending, making cost reduction a significant benefit of virtualization.

The capability to quickly change course and put up a new, isolated partition provides businesses with significant competitive advantage over less flexible methods. The degree of isolation provided by the IBM POWER5* hypervisor translates into reduced risk for new applications and increased opportunities to test changes. Although virtualization may be a newer concept to the UNIX world and many users hesitate to implement test partitions within the same footprint as production, this technology was fine tuned on the IBM mainframe; 40 years of knowledge and lessons learned provide the System p platform with an edge with respect to delivering stability and reliability.

Because deciding how best to utilize virtualization may be daunting at first, we're outlining an easy, four-step approach to implementing and supporting virtualization on System p servers.

Step One

The first and easiest step is to review the number of CPUs on the system. Using this information, create a pool of CPUs and simply allow the hypervisor to distribute the workload most efficiently. This drives increased utilization and can significantly reduce software savings as well. Each LPAR can "cap" or limit the number of CPUs utilized, reducing the number of CPUs that must be licensed for a specific software package. One example might be to have a System p5 570 server with 16 CPUs and create 20 LPARs letting the system distribute workload as needed.

Companies wishing to preserve some CPU capacity for upgrade on demand may choose larger machines with some capacity in reserve. It's also important when choosing the size of memory DIMMs to consider future expansion needs or requests. Smaller DIMMs that fill all available slots may be less expensive at the time, but having some unpopulated slots may be cheaper in the long run.

Step Two

The next step involves virtualizing the I/O by creating a pair of virtual I/O servers. Two virtual I/O servers provide redundancy. We place rootvg on a storage-area network (SAN) and boot from the SAN. This helps save on internal disk space as SAN is more cost-effective and generally provides faster I/O throughput. On average, only two 2 Gb Host Bus Adapter (HBA) cards are necessary. This should handle the workload and paging space requirements of 20 rootvg. Since the majority of bandwidth is consumed during the boot process, there will be unused bandwidth on HBAs unless all 20 LPARs were booting concurrently, which would be unlikely.

Another area of virtualization to consider is network administration and sharing network adapters. This can create a requirement for network backup, which we'll explore further. Examining any system for single points of failure (SPoF) is a crucial step as more components are virtualized. Although the current generation of the System p platform has superior reliability, availability and serviceability (RAS), server consolidation makes it necessary to provide the right availability architecture to support virtualization and consolidation levels.

Looking again at the p570 server running 20 LPARs, we configure two virtual I/O servers for load balancing and redundancy. The system administrator reviews other potential areas of concern (e.g., application availability, failover partitions and ensuring dual power feeds are coming from two different circuits to the system).

Most issues that occur on well-designed systems are done by sleep-deprived system administrators doing work at 2 a.m. Having two virtual I/O servers is one way to protect against that same sleep-deprived administrator typing the wrong command and bringing a single virtual I/O server down. A sample configuration is depicted in Figure 1.

The system with 20 LPARs needs eight HBAs to serve the rootvg for 20 partitions instead of requiring two for each partition. This removes 32 HBAs from the solution and may provide significant cost savings for hardware as well as greater flexibility. For eight HBAs, we reduce the number of PCI slots to four, which allows for greater future growth. This design allows the system to present a single logical unit number (LUN) to both virtual I/O servers per LPAR. Multipath I/O (MPIO) software (such as SSDPCM, PowerPath and HDLM) on the virtual I/O servers is utilized for load balancing. Using MPIO on the AIX* OS/LPARs eliminates a potential SPoF by providing dual paths either to virtual I/O server 1 or virtual I/O server 2. Some installations run all odd-numbered LPARs to virtual I/O server 1 and even-numbered LPARs to virtual I/O server 2. If one of the virtual servers was brought down for maintenance (or even brought down in error), then this prevents administrative overhead dealing with stale disks in each of the LPARs.

Step Three

The third step involves examining the networking aspect of virtualization. This is very similar to the concept of disk. The network's complexity is determined by network requirements. The virtual I/O server provides advanced options for network administration beyond basic networking, including using the Network Interface Backup or using the Shared Ethernet Adapter (SEA).

Network Interface Backup (NIB) provides not only redundant network cards, but it also provides rudimentary load balancing across network I/O adapters. Each adapter is defined and utilized rather than having one defined network card and a standby failover card. Combining this technique with AIX OS features provides network redundancy with fast recovery from network card failure.

If a network failure occurs, the traffic will route to one virtual I/O server. This is a cost-effective way to set up a virtual network as it needs fewer network cards, but requires more administration time as this feature must be set up on each LPAR. Each LPAR must be touched in the event of a network failure to set the paths back to the original virtual I/O server. There are also some limitations such as MTU size of 1500 or smaller and 802.1q (VLAN tagging), which aren't supported with NIB configurations. Figure 2 depicts a typical NIB configuration.

The second way to set up a virtual network is the SEA method. SEA supports VLAN tagging and large packets. It can be set up on the virtual I/O servers and is therefore easier to manage. SEA requires backup network cards on the opposite virtual I/O server, but if the goal in the enterprise is providing the best highly available network and lower cost, this may be the preferred choice. In our example of 20 LPARs with two separate VLANs, instead of having two network cards in each LPAR for 40 total network ports, we reduce the required number to eight network cards. Again, best practices would recommend that a schema to split LPARs between virtual I/O servers be deployed such as odd number LPARs to virtual I/O server 1 and even numbers to virtual I/O server 2. This SEA setup would have four network cards sitting idle waiting for an event but it's a low cost for such high availability. Figure 3 shows one possible configuration.

Step Four

The fourth step looks at virtualizing the non-rootvg or datavg disks. On systems with very high, busy or volatile I/O, this may be the least desirable virtualization. A guideline to keep in mind: If your current system is very busy with I/O, you shouldn't virtualize the disk. System p hardware provides flexibility in this respect. Physical or virtual disk or networking options can be used in any of the LPARs. Start by virtualizing the rootvg to reduce the rootvg HBAs and further virtualize and reduce HBAs for datavg when warranted. Allocate four HBAs for each virtual I/O server in a 20 LPAR system and add the datavg to existing OS LUNs that are already defined for normal workloads (i.e., systems that aren't utilizing significant bandwidth).

Careful analysis of the current SAN environment and port utilization is necessary before planning out the transition, but it's relatively simple to move from physical to virtual when data is on a SAN. When capacity needs outgrow the bandwidth of the virtual subsystem, it's a matter of adding more dedicated HBAs and migrating back from virtual to physical. A hypothetical configuration is shown in Figure 4.

Vital Virtualization

Virtualization provides flexibility and cost savings and allows your System p platform to grow and change with business needs. Hopefully, this four-step process gives you a way to approach the adoption of this important technology. Virtualization is a powerful tool, which provides the flexibility necessary for IT to evolve as business changes.

Fig: 1