New vSphere 5 Licensing & Oracle on vSphere

Following a Twitter conversation between myself and @aus_effendi, Oracle-on-vSphere guru extraordinaire, I thought it might help to create a short post on the implications of Oracle licensing on vSphere given the new vSphere 5 licensing model.

First, I won’t go into much detail on the new vSphere licensing, and I haven’t made my full opinion of it yet so this definitely isn’t a rant on that.  Go here for more info and then come right back!

The basic idea that’s strating to come about, is that it will be a challenge to license dual-socket hosts with greater than 96GB of RAM (assuming Essentials Plus).  Taking one of the new blades with 512 GB of RAM for example, we would need 11-CPU licenses for a 2 CPU host!

Oracle licensing, for those who don’t know, is based upon CPU Cores, not sockets.  For example, a new blade with dual-12 core CPUs would require 12 CPU licenses of Oracle (there is a .5 scaling factor for x86 hardware).  You can then run as many Oracle instances in as many VMs as you want, on that one blade (now up to your vRAM license capacity in vSphere 5).

Now, given that according to the latest price list, a single CPU license of Oracle Enterprise edition is $17,500 (http://www.oracle.com/us/corporate/pricing/technology-price-list-070617.pdf), the blade in the example above would cost $210,000 to license Oracle alone!

Your mileage may vary in terms of the CPU type and memory configuration you plan to buy in the future, but we all know the trend will be towards larger and larger configurations.  It seems this may be one case where paying for the extra hit on vSphere vRAM licenses for vertical scaling will certainly be more favorable than paying for the Oracle licenses required to scale horizontally.

One thing is for certain.  In a world where we already have trouble convincing DBAs and CIOs alike that virtualizing Oracle on vSphere is a great approach (see my previous post http://www.timcurless.com/post/5583656499/profiling-oracle-11g-performance-on-vsphere-with), this licensing duality will only make things more complicated!

Let’s hear your thoughts in the comments!

Profiling Oracle 11g Performance on vSphere with vscsiStats

One of my larger projects recently has been to virtualize a Tier 1 Java application on vSphere for the State of Wisconsin.  This application previously ran on WebSphere 5 and Oracle 9i, on some older physical boxes.  As part of the upgrade we’re moving to WebSphere 7 and Oracle 11g, all 100% virtualized on vSphere.  It was a big sell for the CIO and department, and so far it has been working great.  Our production go live is the upcoming July 4th weekend.

As part of a bunch of Load Testing, I took a few minutes to grab some storage performance data using vscsiStats.  A full dive into vscsiStats is out of the scope of this post, however I will link some great resources at the end of the article.

First, a bit about how I designed the Oracle environment.  Taking advantage of Oracle ASM was really a big plus, however a lot of people were pretty nervous about the extra layer of management/abstraction (myself included).  For the purposes of the design, however, it ended up working pretty well.  In general, this design was really meant to meet a lot of the best practices regarding Oracle 11g on a vSphere platform.

Here’s a look at how one of the Database VMs was designed.  Notice that there are really two ASM volumes, DATA and RECOVERY, that were designed to meet those obvious database needs.  Both volumes are set to External redundancy to really allow our HP EVA 8400 to do the striping. 

Each volume is served by a handful of .vmdk’s on separate dedicated datastores.  These datastores are then mapped to LUNs in a 1:1 fashion.  The disk size, for now, is 250GB, meaning the DATA volume has 4 disks for a total size of 1TB.  In addition, the disks contributing to the DATA volume are on a separate virtual SCSI controller to take advantage of the paravirtualized SCSI driver.

Now for the interesting stuff.  Here is some data gathered by vscsiStats collected over approximately 15 minutes during a load test designed to simulate about 85% of our normal production load.

The first interesting statistic to look at is the average IO size.  Here is the IO Length profile for the DATA disks and RECOVERY disks respectively.  Each member of the group shares a similar profile.

From the graphs we can see the majority of DATA IOs are 8KB.  Conversely, the majority of IOs for RECOVERY are 1KB or 16KB (75%/25% mix).  This tells me to optimize my corresponding SAN disk groups for this IO profile (stripe size).  I believe the EVA has a hard coded 128KB stripe size (which is also the ASM default) so I guess the point is moot.  Unfortunately in my world I don’t have spindles dedicated to this application, but it might be a good justification to management about why we need more disks.

Next, I wanted to take a look at the access profile to confirm what any DBA/SA would expect.  The DATA disk group should be highly random in nature, and could benefit from faster spindles (15k SAS or SSD).  The RECOVERY disk group, meanwhile, is handling logging and should appear highly sequential.  This could be a case for 10k disk or even NL-SATA.  The following graphs confirm.

By and large, we see successive R/W opperations having to travel the maximum positive or negative distance from the previous IO about half the time.  However there is what I would call a “less random” aspect to this as well.

The RECOVERY group (logging) is about as sequential as you can get.

Finally, I wanted to take a look at what effect all of this had on overall latency.  The RECOVERY volume shows sub .5ms latency about 75% of the time, which feels pretty decent to me.

Latency on the DATA volumes is somewhat less than ideal, in my opinion.  About half of the IOs are coming in under .5ms, while the other half are taking longer than 5ms, with up to a quarter taking 15ms.  However with multiple disks serving the volume the burden is eased somewhat.  We’re getting fairly decent average transaction times of about .114s per transaction, so not bad.

EDIT: In retrospect, I was a bit censorious regarding the DATA volume latencies, especially when compared to the previous environment.  If you look at read and write latencies for this disk group individually it was actually quite impressive.

All in all, vscsiStats is an outstanding (free!) tool for profiling your storage workloads.  This is especially great in our case as we justify to the world how virtualizing tier 1 Oracle workloads on vSphere was indeed a great idea!

Please feel free to leave comments, questions, concerns, general theories, etc!

Extra Resources:

http://communities.vmware.com/docs/DOC-10095

http://www.yellow-bricks.com/2009/12/17/vscsistats/

New Network Design but Old Security Models?

We’re full into our core network design here at our Greenfield deployment.  A couple of interesting discussions have come up around the design, particularly in terms of Cisco Virtual Device Contexts (vDC) on the Nexus gear.

First, here’s what we have already sitting on the dock or in the racks:

• (2) Nexus 7010 Core Switches w/ 3 fabric modules and 2 supp’s

• (4) 8-Port 10 GbE blades
• (2) 48-Port 1Gb Copper blades
• (2) Nexus 5020 Agg Switches deployed top-of-rack
• (2) ASA 5580-40’s w/ 4 10 GbE ports ea.
• A handful of Nexus 1000v licenses

A primer for those who, like me, are still fairly unfamiliar with Nexus technology:

The virtual device context idea is really what puts the virtual into Cisco’s virtualized networking (why you still need physical cables for each vDC is beyond me and another story :) ).  As I understand it, a vDC basically allows you to segregate your physical switching into multiple virtual switches, just as if you had bought multiple physical switches to begin with.  Currently the Nexus 7000’s allow up to 4 vDCs, and I believe that will be expanded to eight in the future.

Each vDC requires a pair of physical cables (actually Cisco recommends cabling sufficient to avoid over-subscription based on traffic to other parts of the network) to send traffic between vDCs, as well as a dedicated “peer link” to facilitate heart beat and keep alive.  Again, this is my pretty rudimentary understanding of virtual device contexts.  I’m sure the networking guys can expound on this!

Now, all of that said, here’s what we’re thinking for vDC use cases.

Model 1:

• Default vDC for Management
• vDC2 for Production & DMZ traffic
• vDC3 for Test/Dev traffic

Model 2:

• Same as model 1, but adding vDC4 for DMZ traffic

The upside of Model 2 is that we get added separation for DMZ traffic (potentially more secure), the downside of course being the increase in physical cabling, port density, and the loss of a spare vDC (at least until the allowed number of vDCs goes to 8).

Finally, we come to my question in all of this.  As the Server/Virt guy, I’m starting to ask, “Why are we using traditional security models as we try to foster a private cloud environment?”  Perhaps I’m ignorant about particular security methodologies on the Nexus switches, but it seems to me that creating these security zones (traditional Inside, DMZ, Outside) in conjunction with the ASAs feels a bit archaic.  My thoughts are to use vDCs to split workloads (production, Dev, etc), not as security walls.  There are a host of other features in the Cisco arsenal to handle that (VLANs, VRF, etc).

Now, the hole in my argument, is that while I don’t like the traditional security model in a private cloud, I don’t really have any answers as to what it should be instead.

Thoughts?

Weird Parallax Distortion

I have equipment…in fact, some of it is set to hit the dock tomorrow!  And yet, I still have no idea what will be running on it.  It’s like some weird chicken and egg experiment gone completely wrong.  The list of requirements is short and gray, while the list of assumptions is filling up faster than I can manage.  It’s making this design pretty tough.  Welcome to my twilight zone!

DCF: Day 2

I’m just starting my second day as an IT Admin/Engineer (position still isn’t really defined) with the Wisconsin Department of Children and Families! 

Brief background:  DCF was built by taking the families component from the old dept. of health and family services, and part of the department of workforce development.  DWD still supports most of our IT infrastructure.  As of last week we have equipment on the way, including a pair of Nexus 7000’s for a new core, HP blades and storage, and lots of VMware!  I’m very excited to get the opportunity to participate in a ground-up build.  I’m not entirely sure on what my role will be within this project, but I’ve been told it should center around VMware/Storage.  More details to come!

I’m also very excited for a sit down with a VMware rep today for a whiteboarding session on VMware View as a potential Windows 7 virtual desktop rollout.  We’ve got an office in Milwaukee (about 600 users) still on Novell eDirectory (yuck!).  We’ve been tasked with designing/testing/implementing a virtual desktop solution by year’s end!

Mouse Gestures in Safari 4 Beta

I’ve been forcing myself to play with Safari Beta 4 recently instead of firefox.  The latest version of the browser seems to have a lot of ripped off features that make other browsers good, including a few things ripped off from Apple themselves (cover flow bookmarks).  Lifehacker tells a better story: http://lifehacker.com/5159907/a-hands-on-look-at-safari-4s-crashy-eye-candy

The one thing I’ve never been able to find with Safari are Mouse Gestures (well, that and ad block).  This has been a staple plugin for me in firefox and I had to have it in Safari.  Here’s how I made it happen.

First, you’ll need an app for Mac OS X called “xGestures” (I know, I know, it’s $5, but it was worth it to me).

Next, go through the general setup for XGestures.  I told mine to start when I log in, and my right mouse button activates gestures.

Now, create Safari specific gestures by defining a new custom application.

Finally, create the custom gestures you’ll need.  For starters I created Page Back, Page Forward, New Tab, New Window, and View Page Source.

There you have it.  Easy stuff.

View high resolution

Obamafy