Tag: DevOps
Raj Bhargava (CEO of JumpCloud) and I have been talking about how startups can leverage DevOps and Agile more. We created a conference on DevOps last year for our portfolio companies and it was a huge hit.
DevOps is a movement that we are deeply interested in from multiple perspectives. It’s integral to almost all of the companies we invest in and many, especially in our Glue and Protocol themes, are DevOps driven companies.
In addition to investing in these companies, we are promoting DevOps concepts throughout our portfolio, encouraging learning activities such as the conference, and involved with initiatives such as DevOps.com that is a site to educate the IT community about DevOps.
When Raj asked me to do a Q&A with him on my views around DevOps to help more people understand why I am excited about it, I immediately said yes. If you are interested in hearing my thoughts around how companies can leverage Devops, head on over to JumpCloud’s Q&A with me on DevOps and SaaS and let us know what you think.
And – if you are a VP of Marketing, JumpCloud is looking for a great one.
My partner Ryan McIntyre says that any company doing business on the web should be practicing some form of DevOps. One of the biggest trends in tech today is DevOps, which is closely tied into Agile, Cloud, PaaS, and SDN.
If you remember Gene Kim’s guest post on the importance of DevOp post, or recognize some of the investments we’ve made in our Glue and Protocol themes that are focused on DevOps, such as JumpCloud and VictorOps, this will be a familiar topic.
Last fall, JumpCloud and Softlayer/IBM hosted a DevOps Conference in Boulder for the companies we’ve invested in. At this conference, I heard of an effort to put together a new community site that would pitch a tent big enough for anyone interested in DevOps. This would be a place where technical folks could learn and communicate, where novices could find out more, and where business people could understand why and how DevOps matters.
Alan Shimel, who I have known for over 15 and has been writing for Network World and a bunch of other places was heading up the effort. In typical Shimel fashion, Alan simultaneously put together a top flight collection of content providers while doing a deal and partnering with Martin Logan who had a blog over at the domain, DevOps.com. If you are going to have a DevOps community media site, it is hard to imagine a better domain to for it to live at.
Since that time Alan and Martin have been working hard retooling the old blog into a full-fledged online community e-zine. They launched the site this week with SoftLayer and JumpCloud as founding sponsors. Another one of our portfolio companies, VictorOps is a sponsor and VictorOps CEO Todd Vernon has a regular blog on the DevOps.com site.
The list of contributors to DevOps.com reads like a who’s who of the DevOps world with a goal of having over 100 unique content pieces a month at DevOps.com. But media content is not the only mission. Alan, Martin and team are planning to help amplify the DevOps grass roots efforts around the world through conferences and community events.
I am jazzed to see what Alan makes of it, but I am even more excited to watch the continued growth and influence of DevOps.
Last week our portfolio company, JumpCloud – who is deep in the DevOps market with their automated cloud server management product – hosted the first annual DevOps conference here in Boulder. It was a huge success – we had over 200 people show up and engage in a full day of deep discussions on DevOps.
We are huge fans of the DevOps movement. Similar to how we got involved in the Agile movement early with our investment in Rally Software, we are long on DevOps with investments in companies such as JumpCloud, VictorOps, SendGrid, Pantheon, Authentic8 and others. We see DevOps instantiating the lean startup culture throughout an organization. DevOps promotes short cycle times, automation, and deep integration across a company with the goal of innovating quicker and more effectively against customers’ needs. In short, we view it as a cultural methodology that increases the odds of success for a company.
The day was fantastic, starting with Raj Bhargava (CEO of JumpCloud) and Paul Ford (SoftLayer) kicking things off with a short discussion about what DevOps is. I was next with a quick discussion framing why DevOps is critical to our companies and their customers. From there, we had presentations by Ryan Martens (CTO of Rally) on learnings from Agile, Nathan Day (Chief Scientist of SoftLayer) on the incredible automation at SoftLayer, and a number of great panels from CEOs, CTOs, and VPs of Engineering of DevOps related companies. Three of our portfolio companies – SendGrid, Mocavo, and Gnip – closed the formal part of the day with case studies on different areas of DevOps.
Later, the full group headed to Bacaro for more casual conversations around DevOps. I ended the night at Walnut Brewery with Raj and a few close friends watching the Red Sox lose game two of the World Series to the Cardinals.
The engagement on the topic of DevOps was really powerful. The questions flowed quickly – it’s clear everyone is struggling with how to define DevOps – what it means, who should be involved in an organization, and how to recruit for it. While the word is quickly becoming entrenched, it’s a new category with a wide range of opportunities.
When Raj came to me several months ago suggesting that we should put on a conference around DevOps for all of the Foundry Group, Techstars, and Bullet Time Ventures companies it was easy to be excited about it. I expected about 50 people to participate – it was amazing to look around the room and see 200 really engaged people. I’m proud of Raj and Paul for putting this on and thankful for all of the effort that our companies made to get there and participate!
Two of the themes we love to invest in are Protocol and Glue. We’ve especially been interested in companies that make software developers and DevOps lives better. Some examples include SendGrid, Urban Airship, VictorOps, Pantheon, MongoLab, and Cloudability.
To that end, Raj Bhargava and I created a company called JumpCloud late last year (our eighth venture together). After being involved in hundreds of technology companies, we know that young and fast growing technology companies have little time to devote to the details of managing their server infrastructure. Often, there is a perception that things are fine, until they aren’t. And then much pain ensues.
My partners and I often worry about companies we’ve invested in having enough bandwidth and resources to adequately cover issues of reliability, availability, and security. We know firsthand what that entails, especially as companies hit high-growth inflection points.
Enter JumpCloud. JumpCloud helps DevOps and IT attain high levels of reliability, prevent unplanned downtime, and manage their environments like the big guys, without slowing them down. Watch David Campbell, one of JumpCloud’s other co-founders, explain JumpCloud at TechCrunch Disrupt.
JumpCloud is an agent-based SaaS tool designed for both cloud and physical Linux servers which provides full user management across all your users, all your servers, and all your clouds. JumpCloud also monitors your servers, identifies missing security patches, watches for attacks in progress, and identifies anomalous resource usage. JumpCloud is completely complementary to your Chef / Puppet / Opsworks configuration / automation tools. Think of JumpCloud as taking over server maintenance, management, monitoring, and security once the provisioning tools have done their thing.
JumpCloud closes the gap between what you can do and what you know you should be doing with regard to user management and security of your cloud infrastructure. That means fewer late-night calls, an easier to manage environment, and more reliability for your customers.
Sign up for free today, at JumpCloud, and let Raj and I know what you think.
Also, if you are a DevOps person or senior technical person in your organizations, Raj, Paul Ford from SoftLayer, and I are hosting a private DevOps Conference in Boulder on October 24th. While the event is for Foundry Group, Techstars, and Bullet Time Ventures portfolio companies, we have a few open slots in case a few folks would like to join us. Just reach out to me via email and I’ll get you connected.
We believe great companies can be created anywhere.
When we started Foundry Group, we hypothesized that 33% of our investments would be in Colorado, 33% would be in California, 33% would be “everywhere else”, and 1% would be on Mars.
We still haven’t done the Mars one, but we remain optimistic about the possibility.
Today we’ve announced that we’ve made new investments in LeadPages (a company in Minneapolis) and 3D Robotics (a company in Tijuana, San Diego, and Berkeley). They were joined by our friends in Boulder at VictorOps (across the street from our office) who just raised a $6.5m financing.
Need a drone? That would be 3D Robotics.
Need split-test landing pages, launch pages, sales pages, and other conversion pages? That would be LeadPages.
Need your DevOps life to be less hellish? That would be VictorOps.
I love what we do.
An increasing number of companies we are investors in are focused on DevOps. A year or so ago I read an early draft of a new book titled The Phoenix Project: A Novel About IT, DevOps, and Helping Your Business Win. I really enjoyed it and asked Gene Kim, one of the authors to write a guest post on DevOps. He wrote it a while ago and it has sat in my draft queue waiting for the perfect moment to emerge. That moment is now. Following is a guest post on DevOps by Gene Kim, Multiple Award-Winning CTO, Researcher, Visible Ops Co-Author, Entrepreneur & Founder of Tripwire.
Since 1999, my passion has been studying high performing IT organizations. On this journey, we benchmarked 1,500 IT organization to understand what differentiated the highest performing organizations and allowed them to do what the others only dreamed of. Our findings went into a book that we published in 2004 called The Visible Ops Handbook, which described how these organizations made their “good to great” transformation.
Since then, this journey has taken me straight into the heart of the DevOps movement. Although I initially dismissed DevOps as just another marketing fad, my friend John Willis corrected me, in the way that only true friends can do, saying, “Don’t be dense. DevOps finally proves how IT can be a strategic advantage that allows a business to beat the pants off the competition. This is the moment we’ve all been waiting for.”
In that moment, I saw the light. Over the years, I’ve come to believe with moral certainty that everyone needs DevOps now, especially software startups where the successful execution of Development and IT Operations preordain success or failure.
Today, we can see how DevOps patterns enable organizations like Etsy, Netflix, Facebook, Amazon, Twitter and Google to achieve levels of performance that were unthinkable even five years ago. They are doing tens, hundreds or even thousands of code deploys per day, while delivering world-class stability, reliability and security.
DevOps refers to the emerging professional movement that advocates a collaborative working relationship between Development and IT Operations, resulting in the fast flow of planned work (i.e., high deploy rates), while simultaneously increasing the reliability, stability, resilience of the production environment.
The culture and practices that enable DevOps to happen cannot be delegated away. In a growing startup where teams start to specialize and multiply, the chaos of daily work often starts to slow down the smooth flow of work between Development and IT Operations, sometimes even resulting in outright tribal warfare.
In this blog post, I’ll describe what this downward spiral looks like, and what everyone in the company must do to break this destructive pattern and ensure that Development and IT Operations work together in a way that creates such a competitive advantage that it may almost seem unfair.
Why Everyone Needs DevOps
There is currently a core, chronic conflict that exists in almost every IT organization. It is so powerful that it practically pre-ordains horrible outcomes, if not abject failure. It happens in both large and small organizations, for-profit and non-profit, and across every type of industry.
In fact, this destructive pattern is the root cause of one of the biggest problems we face as an industry. But, if we can beat it, we’ll have the potential to generate more economic value than anything we’ve seen in the previous 30 years.
I’m going to share with you what this destructive pattern is in Three Acts, that will surely be familiar to you. (You can get the whole story in my book, The Phoenix Project: A Novel About IT, DevOps, and Helping Your Business Win).
Act I begins with IT Operations, where we’re supporting a large, complex revenue generating application. The problem is that everyone knows that the application and supporting infrastructure is… fragile.
How do we know? Because every time anyone touches it, it breaks horrifically, causing an epic amount of unplanned work for everyone.
The shameful part is how we find out about the outage: Instead of through an internal monitoring tool, it’s a salesperson calling, saying, “Hey, Gene, something strange is happening. Our revenue pipeline stopped for two hours.” Or, “the banner ads in my market are being served upside down and in Spanish.”
There are so many moving parts that it takes way too long to figure out what caused the problem du jour, which means we’re spending more and more time on unplanned work and increasingly unable to get our planned work done.
Eventually, our ability to support the most important applications and business initiatives goes down. When this happens, the organization suddenly finds itself unable to achieve the promises and commitments made to the outside world, whether it’s investors, customers, analysts or Wall Street.
Promised features aren’t delivered on time, market share isn’t going up, average order sizes are going down, specific revenue goals are being missed…And that’s when something really terrible happens.
In Act 2, everyone’s lives gets worse when the business starts making even bigger promises to the people we let down, to compensate for the promises we previously broke. Often, the entire organization starts dreaming up bigger, bolder features that are sure to dazzle the marketplace, but without the best grasp on what technology can and can’t do, or fully realizing what caused us to miss our commitments in the first place.
Enter the Developers. They start seeing more and more urgent date-driven projects put in the queue, often requiring things that the organization has never done before. Because the date can’t be moved (because of all those external promises made), everyone has to start cutting corners.
Development must focus on getting the features done, so the corners that get cut are all the non-functional requirements (e.g., manageability, scalability, reliability, security, and so forth). This means that technical debt starts to increase. And that means increasingly fragile infrastructure in production.
It is called “technical debt” for a reason—because technical debt, like financial debt, compounds.
When technical debt begins to accumulate, something very insidious starts happening. Our deployments start taking longer. What used to take an hour now takes three hours, then a day, then two days—which is okay, because it can still get it done in a weekend. But then it takes three days, and then a week, then two weeks!
Our deployments become so expensive and so difficult that the business says that we have to lengthen the deployment intervals, which goes against all our instincts and training. We know that we need to shrink the batch sizes, not make them bigger, because large changes make for larger failures.
The flow of features slows to a trickle, the deployments take even longer, more things go wrong, and because of all the moving pieces, issues take even longer to diagnose. Our best Dev and Ops people are spending all their time firefighting, and blaming each other when things go wrong.
I’m guessing that most of you can relate to at least some portions of this story? As I said, this happens both in large enterprises and growing startups alike. In my fifteen years of research in this area, I’ve found almost all IT professionals have experienced this cycle.
Act 3: How DevOps Breaks Us Out Of Our Downward Spiral
We know that there must be better way, right? DevOps is the proof that it’s possible to break the core, chronic conflict, so we can deliver a fast flow of features without causing chaos and disruption to the production environment.
When John Allspaw and Paul Hammond gave their seminal “10+ Deploys Per Day: Dev and Ops Cooperation at Flickr” presentation at the 2009 Velocity Conference, people were shocked and amazed, if not outright fainting in the aisles at the audaciousness of their achievement.
It wasn’t a fluke. Other organizations such as Facebook, Amazon, Netflix and the ever-growing DevOps community have replicated their performance, doing hundreds, and even thousands, of deployments per day. DevOps is not only for large, established companies. It’s for any company where the achievement of business goals rely upon both Development and IT Operations. These days, that means almost every company.
We all need to be putting DevOps-like practices into place. This is why Kevin Behr, George Spafford, and I wrote The Phoenix Project: A Novel About IT, DevOps, and Helping Your Business Win.
A novel, you might ask? How is a novel going to solve my problems?
As a friend once told me, “Before you can solve a complex problem, you must first have empathy for the other stakeholders. And story-telling is most effective means of creating a shared understanding of the problem.”
Dr. Eliyahu Goldratt demonstrated the power of a novel as a teaching tool through his book, The Goal: A Process of Ongoing Improvement. It’s a novel written in the 1980s about a plant manager who has 90 days to fix his cost and due date issues or his plant will be shut down. When I read this book nearly 15 years ago, I knew that this story was important, and that there was much I needed to learn, even though I never managed or worked in a manufacturing plant.
It isn’t an overstatement to say that The Goal and Dr. Goldratt’s Theory of Constraints changed my life—in fact, it probably was one of the biggest influences on my professional thinking. For eight years, my co-authors and I wanted to write The Phoenix Project, because we all believed that IT is not a department, but a strategic capability that every business must have.
As you can imagine, I was incredibly honored and thrilled when Jez Humble, author of the award-winning book Continuous Delivery recently told me, “This book is a gripping tale that captures brilliantly the dilemmas that face companies which depend on IT. The Phoenix Project will have a profound effect on IT, just as The Goal did for manufacturing.”
Prescriptive Steps
For those of you are looking for some places to start your DevOps journey, here are my three favorite DevOps patterns:
- Make sure we have environments available early in the Development process. Enforce a policy that the code and environment are tested together, even at the earliest stages of the project.In the ideal, IT Operations is able to create an environment (that is, everything except for the application code: databases, operating system, networking, virtualization layer, etc.) with one step. That can be as simple as copying a virtual machine, or as complex as an automated build system that generates the environment from scratch (e.g., puppet, chef, etc.)Furthermore, use the same build mechanism to build the Production, Test and Dev environments at the same time. If we modify the Agile sprint policy so that instead of merely having shippable code, we have shippable code and the environment that it runs within, we’ll have done code deployments many, many times when it’s time for the real-life production deployment.
- “Wake up developers up at 2 a.m. when they break things.” Yep, you heard me. This quote came from Patrick Lightbody, the CEO and founder of BrowserMob. He continued, “When we woke up developers, we found that defects got fixed faster than ever.”The goal is to shorten and amplify feedback loops, and to bring Development closer to the customer experience. In DevOps work streams, developers often deploy their own code, and fixes forward when things go wrong. By doing this, developers can see the consequences of their decisions and actions.(Note the symmetry here: the previous pattern #1 about making environments available early is all about embedding IT Operations into Development, while this pattern is about putting Development into IT Operations.)
- Create reusable deployment procedures: When every deployment is done differently, every production environment can become different, like snowflakes. When this occurs, no mastery is ever built in the organization in procedures or configurations. As Luke Kanies said, “If your infrastructure is special, you’re doing it wrong.”To make this reality, create reusable user story for IT Operations, such as “Deploy app into high availability environment,” which then goes on to define exactly the steps to build the environment, as well as how long it takes, what resources are required, etc.By doing this, we codify the deployment and engineering procedures requires to build reliable, resilient and properly configured environments, and can then factor that into our planning processes, such as in the PMO.
Conclusion
If you enjoyed this taste of DevOps and believe it can help achieve your goals, “The Phoenix Project” is available now, or you can download a free 170 page excerpt of the book. And of course, you can always find the latest writings on DevOps at the IT Revolution blog, where you can get our free whitepaper “The Top 11 Things You Need To Know About DevOps.”
Long live DevOps!