Tag Archive: VMware

I am sitting on the flight back home from VMworld and listening to podcasts from the conference. Seems the biggest deal was EVO:RAIL/RACK coming out. At first when I heard about Nutanix two years ago, I didn’t see them being an enterprise ready but man has the area grown.

Enter in Simplivity, I met with the local reps about 4 or so months ago and got to see what they are doing in this hyper-convergence market (like Nutanix and now EVO) and got to say I was impressed. One because they used Dell servers, which is a big partner of ours, but importantly all the functionality they have built in to their product. I had Nutanix sponsor our OKC VMUG last month and next Simplivity on October 24th, so it will good to see the differences. I highly recommend you to research them both so I don’t show bias here.

But why would you go the hyper-convergence route in your data
center? For me lately, I have been tasked to architect building blocks for both server and VDI environments. I totally see these three (Nutanix/Simplivity/EVO:RACK) be the path for VDI. One, each bring their own storage solutions so I can keep the IOPs off our primary storage (cause man Oracle is crazy needy!). And two, they make it easy to say “you need 100 VDIs, you need one of these..” and grow accordingly.

This doesn’t mean they suck at server VMs but I don’t need additional storage for those right now. Most of us already have that, but if you are building new look at them. Save rack space, power, cooling, and network ports by going this way in your virtualization environment. Each have limitations and functionality that the other don’t have but don’t want to start a vendor war.

Now my question, would you switch?


This past Sunday, my phone was blowing up with email alerts.  But what caught my eye was the alert that only happens when vCenter Service starts:


* For those that didn’t know, at one point VMware started this much hated vRAM tax on vSphere 5.0 and as you can see we still have some hosts/licenses tied to this model that they did away with in vSphere 5.1

When I finally saw this alert over and over, I knew that the vCenter Service was automatically restarting almost every 10 minutes.  So after logging in I saw the following in the Windows Event Log:


As you can see that it states “..database ‘vcenter’ because the ‘PRIMARY’ filegroup is full…”  so I checked the SQL server that was created just for VMware related databases was indeed out of space.  Luckily it’s a VM so I just added more space to and then restarted the VMware VirtualCenter Server service and all was fixed within 10 minutes.

One last thing to check, if you have vCloud Director as well, is to reconnect the vCenter inside of vCD or you may not be able to provision new VMs or open Console:


OUIT Shared Services – 2013

This year has been packed with things that we’ve been doing in S2 at University of Oklahoma.  We went production with both data centers in OKC and Norman being a stretched cluster.  Brought up vCloud Director to give our users self provisioning and better visibility into their environment.  This meant we had to get storage, both block and file, working across then we even upgraded the controllers for the Dell Compellent SAN.  We were able to do this with little to no downtime to our customers which proved what we designed it to do.

Now was it all easy and full of great smelling flowers?  No way.  I would love to say we can engineer an active/active data center with no issues.  But we have enough great engineers to quickly resolve or remedy a problem.  I’d say over all phase 1 of S2 is going pretty well.  If it wasn’t I doubt I would have been able to present over it on a vBrownBag podcast or a session at VMworld 2013 in San Francisco.  Now that was exciting, not everyone can say they spoke at a conference that is attended by 22,000+ people from around the world.

Phase 1 included:

  • Two Active/Active Datacenters (OKC and Norman with Tulsa being a separate data center for now)
  • 4 Dell Compellent SANs (2 each DC) with 800 TB RAW all together – largest environment in Oklahoma currently
  • 2 EMC Isilon NAS (1 each DC) with around 250 TB that can replicate to each other
  • 4 Cisco 7K’s (2 each DC) and multiple 5K’s and 2K’s that are linked together using either our dark fiber between us or OneNET’s path.
  • Juniper SRX firewalls
  • Palo Alto for IPS/IDS
  • 16 vSphere ESXi 5.1 general cluster nodes (8 each DC/Dell PowerEdge blades)
  • vCloud Director 5.5/vCOPS 5.7
  • View 5.2 (OKC only for now)

Phase 2 will be finishing the upgrade of our VMware environment to vCloud Suite 5.5 and seeing what vCAC 6 could bring to the table.  More training is already scheduled for our Operations and Design teams on vCloud Director, vCOPS, View and CommVault.  Simpana 10 backup is installed and configured but slowly rolling this out so we can finish up transitioning this into Shared Services as an offering to our customers.

It will be a busy year for us in 2014 but it will be a ride.. time will tell if it’s a great, good or heck of one!

Many thanks goes out to:

  • My coworkers, specifically David Stricklin (@strickfila) for backing me up when needed/keeping me in check and our VP David Horton (@hortonhearsyou) for doing all that he does for OU/OUHSC.
  • Sean O’Dell (@theseanodell) from VMware on always assisting me the OKC VMUG, keeping us up-to-date with VMW products and co-presenting at VMworld.
  • The vBrownBag community (@vbrownbag/#vbrownbag) like Jon Harris (@thevcacguy), Damian Karlson (@sixfootdad), Cody Bunch (@cody_bunch) and many others.  Check them out at http://www.professionalvmware.com
  • The VMUG community (@myvmug) as a whole and there’s not another one like it.
  • Also all past and current vExperts to which I’m thankful to say I am 1 out of 581 to be named in 2013.

If you do have Twitter, I highly recommend following these people as well as our Shared Services (@ouits2) and OKC VMUG (@OKCVMUG) pages to see what is going on for 2014.

There are several podcasts that I recommend besides just vBrownBag:


Now that VMworld is over and I’m back to work, it only seems right to write a take away about it.

First, my buddy and coworker David Stricklin went again as well this year so this marks his 4th and my 5th attendance.  Also our friend from the north campus, David Wisby, was also able to go so we definitely were able to “divide and conquer” the sessions as well as meet up for the parties.  Being a vExpert and VMUG leader there were a couple that only I could go so I was glad that both David’s (I’ll start calling them by last names) were able to hook up and have fun.

This was also the first time I went as a presenter.  My session was with Sean O’Dell who works at VMware and helps me with presentations/content for the OKC VMUG.  We had 116 register and 82 actually attend.  Not a bad turn out to hear what we are doing here at the University of Oklahoma with Shared Services.  It was great to get follow up questions and to hear of other universities starting to do the same as us.  The feedback was good except for the “it was boring and stop with the inside jokes” one.  I guess if you leave less than 10 minutes into the session you might think that and I apologize for my nervousness that I eventually was able to get out of the way.

The vExpert and VMUG parties were a great way to meet others in the two communities and nice to see that CEO Pat Gelsinger, COO Carl Eschenbach and then heard that former CTO Steven Herrod showed up after I left.  I highly doubt any other community would get that treatment but they need to.  I am definitely looking forward to next year and may try to do a follow up presentation on how we’ve progressed (hopefully the move from vCD to vCAC).

If you are in the IT field, be it virtualization, network or whatever I highly recommend following these vExperts on Twitter.  A lot of good info across the board and around the world. I will try to do a more in depth write up about the sessions I attended and the take-aways but now I have to go cry for installing and getting vCloud Director going for production a month before VMworld only to hear that it’s eventually going away and being cut up to get the multi-tenate in vCAC and other features else where.  Don’t mind the tears on the keyboard.



Well, today was the day that the vExperts for 2013 would be announced. I applied over a month ago basically for the conferences in OKC that I spoke at, the Shared Services initiative that I have been helping on at work and bigger reason, the OKC VMUG.

I really didn’t think I would be accepted the first year of trying, a vExpert is not something you just earn for saying you do virtualization. It’s a community based upon helping the community grow. It’s not a club that you can say I was once a member and always a member, you have to keep doing your “job” as a virtualization community member or you most likely will not get the vExpert title the following year.

So without further ado, the link to the close to 580 vExperts for 2013.

Thank you all!

vCOPS for Monitoring

We started “testing” vCOPS a few months back because of the projects we have going on. I am always happy with doing an install that requires me to just import an OVF and give some IPs or set to DHCP and click finish. Of course there’s some settings in the app that you have to do but that’s not out of the norm.

Now here’s where some people have a difficult time understanding. Why does VMware recommend letting it run for 21+ days before going with the stats it gathers day one? vCOPS doesn’t just pull metrics from vCenter and trust it. As an admin I would want it to bake for at the very least a month and still monitor the changes a monitoring tool tells me.

The changes in version 5.6 are improving the GUI for users and giving us more power to create groups. VMware is also giving away a foundation edition to all vSphere customers. Now yes, it is limited but it gives you the opportunity to try it out against all your hosts and VMs instead of just one host and limited number of VMs like some other vendors are doing.

If you are also a View shop, check our vCOPS for View. It gives you info from the endpoint to the host to the connection manager to the VDI. I was able to pull stats as to why a user was experiencing slowness and point it to the fat client getting packet loss.

I will go more into detail about our experience but want to give a shout out to Sean O’Dell from VMware for coming out and showing us the new features as well as giving us ideas on what else Hyperic can monitor with vCOPS.

Today was suppose to be an easy day of racking and setting up a Isilon in our Norman lab.  Things were going off without a problem except for the last rail, screw wouldn’t go in.  But that was an easy work around.  But back to the long subject in the post.  I needed to get our blades in our Dell M1000E up and going for some testing.  Storage isn’t ready just yet as we have been waiting on connectivity to finish up since we can’t run iSCSI on wifi.  🙂  But I did have a local 146GB drive in each blade that’s not being used since I installed ESXi onto the SD card.  Easy right?  Just create a datastore like you’ve done hundreds of times, no problems.  Or so I thought.

Going to the first blade after connecting directly to the host with the vSphere Client and selecting the LUN in the Storage tab and create Datastore.  All is going and thinking that this was going to be a homerun.  About 5 minutes after it’s spinning at “loading..” on creating the VMFS5 partition the vSphere Client just bombs with an error that I pressed OK too fast to read.  Thinking it was network related, again our connectivity team is still finishing up their part as well as getting the firewalls and DNS/DHCP stuff knocked out, we start down a path of trouble shooting the network.  We noticed some NAT translations going on that didn’t need to be after adding the hosts to our lab vCenter.  Got that resolved but the issue was still there.  Now thinking it’s the firewalls and get Juniper support on the line.  They notice some issues but nothing really related to the ESXi dropping.

Now here comes the part I SHOULD have noticed on the first crash.  The error of Call fails for “HostDatastoreSystem.QueryVmfsDatastore- CreateOptions” for object “ha-datastoresystem”  Yea.. um.. what? That wasn’t what I thought it said the first couple times it crashed.  But my stupidity of thinking it was a network problem (cause it’s always a network problem.. right storage guys? or was that “it’s always a storage problem” from the network guys.. I forget).  Simple Google search and guess what, there was some Dell partitions on that local drive.  D’oh!! Deleted those and boom, hosts with local datastores done.

Thank you Eversity Blog for typing something up 2 years ago to make me face palm myself.


Well, started thinking I need to get into the blogging world. With all the projects going on it’s really the best way to keep up with them all.

Back in November, I started up the OKC VMUG again. The group had been dead for over 2 years. Within the last 3 years I have been taking over more of a virtualization subject matter expert here at OUHSC. So I have been craving the opportunity to meet others in the same field and bounce ideas around. This is why I decided to become the leader and get the VMUG going to not just learn more myself but also help others that their job may not get to afford to send to VMworld or training. So this is where I begin in the blogging community.