[ Content | View menu ]

Politics and Analogies

Mark Mzyk November 6, 2012

Today is Tuesday, November 6th, 2012. Election day in the United States. Last night, my local PBS station aired the Frontline episode The Choice, which is about the biographies of Barack Obama and Mitt Romney and how they found themselves at this point, running for election to be the President of the United States. The episode was fascinating and made me think deeply. I thought I’d share some thoughts here, with an analogy that came into my head today. Like all analogies, it isn’t perfect, but I think it is illustrative.

First though, if you’re a citizen of the United States and you are allowed to do so, go vote. If you dislike Obama and Romney, you can vote for the third party candidate, Gary Johnson, if he is on the ballot in your state (he is on the ballot in most states). You can even skip the presidental election if you so choose and vote for the other offices. Local elected officials have a huge impact on your life – you should vote.

Now that the PSA is out of the way, some thoughts and the analogy. If you have time, I recommend you watch the Frontline episode. It presents a look at the candidates and what has shaped them that I hadn’t seen anywhere else. The episode clocks in at around two hours.

What struck me, after watching the episode, was how Obama and Romney both faced one major criticism from their time in office (Obama for the Presidency and Romney for the Governorship of Massachusetts) and that was that neither is good at connecting with other lawmakers. Neither one has the back slapping, make you feel good personality that the likes of Lyndon B. Johnson had in spades that makes bipartisanship easier. This implies that even if Romney replaces Obama the personal dynamics of Washington aren’t likely going to change.

For examples of both failing to reach across the aisle, see Barack Obama failing to get a single Republican vote for health care or failing to achieve a debt ceiling deal. While you can argue that Republican’s bear some (perhaps even a lot) of responsibility, some blame does go to Barack Obama for not being more effective in reaching out. As for Mitt Romney, I was surprised to learn that while governor of Massachusetts he vetoed over 800 bills. That’s a phenomenal amount of bills to veto over a four year period. Many were apparently then overturned by the legislature. Romney’s biggest achievement came when he decided he need a big achievement so he worked to pass the health care bill, which was something Democrat’s wanted to pass. In passing this Romney was very effective and seemed to have learned a lesson from all the vetos. The amount of vetos is still staggering though.

I was also struck by how much Romney’s business experience has shaped him. He appears to lead his life as if it is one large consultancy, where he comes in as the outsider, prescribes the solution and puts it into place. Romney has been very successful at doing this and I gained a respect for him that I hadn’t had before. That said, there are caveats to his success. His business experience is in the consultancy/investment world, which is not the same as building a product business. I don’t know that experience in one of these worlds translates into the other (or the Presidency).

The other knock on Romney running his life as a consultancy is that he changes his message to fit what he thinks will make the sell. It remains very difficult to figure out what he truly believes, although I think he does have beliefs. He just hasn’t communicated them well. Conversely, Obama has not done a good job communicating his plans for a second term that well either.

On to the analogy I’ve promised:

Barack Obama is Apple, while Mitt Romney is Microsoft.

Why is this?

Obama is Apple because he has had a career defined by huge success and has a massive signature product (Health Care) much as Apple has the iPod/iPad. Obama is very stylized, like Apple products. They do one thing and they do it well, generally. However, Apple has its duds too (such as the Newton and Mac Cube). Obama has experienced his own recent failures, after a string of successes. Can Obama come back like Apple did after Steve Jobs returned?

Romney is Microsoft because he’s achieved huge success  – running Bain Capital, becoming Governor of Massachusetts, rescuing the Salt Lake Games. From these successes he’s built up a massive amount of wealth, which Romney uses to sustain himself while he tries to generate new success, like Microsoft uses the money it has built up from Windows and Office to try and enter new markets. Romney is trying to be everything to everyone and generally failing to achieve new success. Like the Microsoft Surface Tablet, Romney generates interest for a while, but then it’s not clear if that short term interest will change into long term success.

This analogy isn’t perfect, as both men and both companies are complicated. I’m sure it’s possible to to come up with an analogy where Romney is Apple and Obama is Microsoft. I do think the analogy I’ve laid out fits though.

I don’t mean this post as a knock on either candidate, but as a reflection of what I’ve come to see in them and what I learned from the Frontline episode. I am an Obama supporter, but watching the Frontline episode gave me an appreciation for Romney and what he has achieved.

And now that this is done I’ll be going back to my regularly scheduled tech topics.

A note on comments: no vitriol please. I welcome all constructive criticism, but attacks on other’s character or uncalled for comments will be policed and deleted. Fair warning, since I don’t know what type of reaction this post will generate.



General - 0 Comments

Surge 2012 Recollections

Mark Mzyk October 8, 2012

Being one of the representatives of Opscode at Surge 2012, here are my (slightly belated) thoughts from having attended the conference.

The list of sessions can be found here.

The opening keynote was an analysts of Forrester who presented four general trends that he said made scalability and performance more important than ever. This talk was forgettable. It was entirely non-technical and spoke about items such as wearable computing becoming a trend that Forrester saw coming into its own in 2017. I think the intent was for the talk to give the entire weekend a theme and relate it back to the customer. That is my charitable interpretation, however. Apparently the analyst, Mike Guiltieri, is the person responsible for coining no-ops, but this talk didn’t touch on that at all.

I attended the Build Your Own Database: BerkeleyDB Java Edition at Yammer. Unfortunately this talk did not live up to its lofty title. I was hoping for a talk on DB theory and how Yammer used that to build a database system. Instead this talk was about why Yammer opted to use BerkeleyDB and build tooling around it for their use case. Their use case turned out to be that they didn’t want to shard Postgres. Yet they basically built a sharded system on top of BerkelyDB and the speaker admitted as much in the Q&A period. I assume Yammer had their reason for building on top of BerkeleyDB, but it wasn’t clear from the talk that sharding Postgres might not have worked out just as well, with more community to rely on.

Bryce Howard’s talk, with a long title and poor comparison of the internet to pipes and water, was a good talk. It focused on the the HTTP and SSL stack and the handshake process. Bryce went through how he debugged the network layer and looked at the handshakes that were occurring. By analyzing the process, Bryce tuned his system so it went from 3 or 4 handshakes to 1, shaving seconds off of each request his system had to make, bringing the request time in-line with what his customers needed and saving him from having to look at other costly options. It was a good talk to highlight how understanding a domain can bring significant improvement into a system.

Tom Daly, of DynDNS, did a talk on AnyCast. This was interesting because it dove into the deep end of DNS and showed how DNS providers (and others who might want to take advantage) are fooling the DNS network by assigning the same IP to multiple machines and taking advantage of routing rules to ensure that DNS lookups go to the closest geographically located machine. It was a technically in-depth  talk that again showed how mastery of a domain can give significant performance improvements. It also proved to me just how little I know about operations and specifically about the network.

I opted to give the Insights into Entreprise Application Management talk a go. It was forgettable. The talk might have been relevant 10 years ago, but not really today. The best insight was that garbage collections cause a process in processing and some tuning that could be done to mitigate this as much as possible.

Artur Bergman in Mysteries of a CDN Explained went deep into the workings of, obviously, CDNs. The long and short of it is that CDNs provide value by acting as close caches to the customer and it’s worth it to cache all the things. Even if someone doesn’t have a lot of content they want to cache, CDNs can still provide value with serving as a single entry point into a network, where if many requests are coming in they can be routed through the CDN which can be a throttle, sending one request on, caching the result for a short time (say a minute) and serving the answer back out for that one minute. This can be a life saver during a high load event. The most entertaining part of Artur’s talk was where he claimed AnyCast DNS was shit and Tom Daly was sitting in the room. During Q&A Tom would get Artur to admit his tests were not the best, but Artur wouldn’t fully back off his statement.

During Lightning Talk, Theo Schlossnagle taught everyone how to tie their shoes, someone showed us what beer brewing looks like, and Bryan Cantrill led a Linux forensics session tracing the history and use of a vestigial Linux program ta that still lives on in the operating system, but would no longer ever be used for anything. The lesson: software hangs around much longer than you probably realize.

For day two, the first talk was Paul Hinze and Tony Pitluga of Braintree talking about how they achieve HA in their datacenter. It was interesting in that they talked about the looks they use and how the avoid traditional alerting of X has gone outside the normal range. Instead they prefer a holistic view of their machines and have written custom tools to provide this. The main tool isLitmus Paper, which is how the configure a nodes health. The way health currently works is by examing multiple attributes on a node and using weights assigns an overall health of a node. The load balance can then monitor this number and send traffic only to health nodes. They did mention that they had to do some tuning to this system to avoid the thundering herd problem. Currently the system is simplistic in that it doesn’t take into account many factors in determining a nodes health, but they plan to keep refining the system as they learn more. In addition to Litmus Paper, they make use of several other open source tools, such as Pacemaker, to make the system work. A big take away from this talk for me was that we should consider looking at our monitoring and figuring how how to put in place a more holistic system. Just because a node is under load, for instance, doesn’t mean something is wrong and needs action – or if it does, why can’t system adjust itself? It will take baby steps, but I think there is progress that can be made here.

Bryan Cantrill and Brendan Gregg were up next. They spoke on DTrace and how they used it to debug system problems at a very low level in JoyentOS. This was one of the best talks of the conference and it was amazing to see them work in a system they clearly knew intimately. They did reveal places where the OS was the problem, but I think it is still safe to assume, on a day to day basis, that the OS is not the problem. It did highlight how as the world gets evermore interconnected and systems are expected to be ever faster problems in the OS can have a ripple effect to the application level.

This talk had one truly fantastic component to it: since Bryan and Brendan were working at such a low leve, they were graphing everything. Every single call through the system (in this case to access a file) and how long it took. They they graphed it. But they didn’t just look at average time. They purposefully created graphs that would plot all the points and the would highlight outliers. This showed things such as a system where the average call was sub-seconds, but them outliers would take ten seconds or more. Investigating the path of the outliers showed flaws in the system. Another graph they showed was the heat map. This was great in that it showed roughly all the calls in the system, but the ones that took the most timer were a darker color. At a glance this showed the average performance of the system but also allowed potential outliers to stand out. A simple average graph loses all this data. This opened my eyes again to how we’re generally approaching monitoring wrong. We need to step back and look more at system’s holistically. This is next realm that tools need to evolve into.

The next talk was Xtreme Deployment by Theo Schlossnagle. I’ll be honest, I don’t remember a lot from this talk, except I know I enjoyed it at the time. I do recall thinking that Circonus is a really nice tool and puts Nagios to shame. Given that you have to pay for Circonus, there’s probably a reason it is light years beyond Nagios.

The next session after lunch was Pedro Canahuati from Facebook, but I only caught the tail end of it where he was describing some of Facebook’s monitoring systems. I was late because I opted to explore Baltimore some after lunch. I hate travelling to a city and then not seeing any of it and Baltimore had a book fair going on that day just a few blocks from the conference.

Matt Graham of Etsy gave a talk. Basically, they deploy a lot. This reduces risk because changes are always small, so they can pinpoint bugs quickly and then deploy quickly to fix them. They measure bughours – how long is a bug alive. With continues deployment they can cut this down drastically because they’re always deploying.

Baron Schwartz gave a talk on Automated Fault detection, but at this point I was pretty fried and had quit taking notes. My apologies to Baron, but I don’t recall much from this talk, other than thinking it was all stuff I had heard before (which, I should point out, doesn’t mean it wasn’t good – just because I’ve heard it doesn’t mean everyone at the talk has, or that I couldn’t stand to hear it again).

The talk concluded with Theo giving a final keynote and parting words, which if I recall correctly was that we should look at other professions and see how they handle failure, because much of what we’re learning in DevOps has been done before. We just need to look past our own walls and see what the larger world is teaching us. My apologies to Theo if that isn’t what he said, but that’s what I’m recalling right now, although I might be mixing it up with Chris Brown’s talk from Velocity Europe that I just watched.

All in all, Surge was a great conference for showing me just how much I don’t know and highlighting directions the industry is moving in, which is both more specialized and more generalized at the same time. It’s clear that as systems get ever larger and more complicated that communication is going to become ever more important, both from a human to human and from a machine to human perspective. DevOps is just the start.

For another perspective on Surge 2012, @obfuscurity’s write up is here.

General - 0 Comments

Triangle DevOps

Mark Mzyk June 23, 2012


DevOps is not a role. It is not a title. Fundamentally DevOps is an idea and a way of behaving. It is the belief that development and operations are not domains that have a wall between them, but that development and operations work best when both sides share ideas and understand the concerns and patterns of the other. In practice this can take many forms. Just as everyone interprets and implements agile practices differently, it is the same with DevOps. There is no right or wrong way, because the only way is continually striving to get better.

Triangle DevOps

Sometimes an opportunity comes along and you don’t think much about it. You jump in and start working before you’ve realized what you’ve done.

That’s the situation I find myself in now that I’ve taken over as organizer of the Triangle DevOps meetup. Without much thought I agreed to take over the group. Now I have an obligation to do my best to make the group succeed, because the Triangle has a wealth of DevOps talent and it needs a place to showcase it.

I will do my best to make the group a fun and inclusive place for the exchange of ideas and the development of talks. The group is going to meet on the third Wednesday of the month at 7pm and is going to rotate between three hosting sites: Bronto in Durham, Teradata/Aprimo in Cary, and WebAssign in Raleigh.

DevOps has two parts: Development and Operations. Each part should be interpreted as broadly as possible. I welcome talks on anything that can even be tangentially related to DevOps. The main criteria for a talk is simple: does it contain an idea that will help to make us better? It is only through exposure to new ideas and new ways of thinking that we grow. I can’t know which idea will help someone grow, so the best strategy is to present as many as possible.

It is also a truth that anyone can have an idea. While it’s great to listen to polished speakers present ideas and we should all listen to them, it isn’t true that they are the only ones with good ideas. Any one will be welcome to present at Triangle DevOps, even if they’ve never given a talk before. Only by giving that first talk can someone start on the road to becoming a polished speaker themselves.

Most of all Triangle DevOps is about always getting better. Better talks, better ideas, better community. Talks and ideas follow from community. Without community there is no Triangle DevOps. I invite you to come out and participate. Share your ideas. You might learn that small trick that makes your life easier. And that would be great. That would make the group a smashing success.

General - 0 Comments