Software Development and Cloud Computing

Cloud computing is a relatively new term for what has been around for a decade – leveraging virtual environments within the firewall  and beyond it. For many software engineering teams, we’ve been in the cloud since its inception and will likely be drivers for it’s adoption going forward.

HOW DID WE GET HERE?

Previous to virtual environments, the infrastructure to support development efforts were physical. Thus, software engineers had their development tools loaded on their respective workstations and integration, test and production environments were made up of servers on the network. Thus, for every new infrastructure came the procurement of hardware. Not to mention how much of a corrupting affect development tools and projects have on a developer’s workstation.

If you subscribe to the definition of a cloud environment as being any that is virtual – even those behind the firewall – then the first iteration of a private cloud for software development came with the emergence of using virtual environments for development workstations. This had a mammoth effect on development efficiencies and IT governance.

As an example, in 2001, I was hired as a consultant to help a client develop the next generation of their commercial e-Discovery product. The suite of applications were complex with many moving parts and technologies. It took the average developer 1-2 weeks to get their development environment up and running. There were limited environments to promote code, and continuous integration wasn’t even a dream.

After about 9 months I left, as consultants often do, but by 2005 I was hired back again for yet another next generation development effort. This time, the developer workstations were based off of a base VM. Getting up in running as a developer took hours instead of weeks. The fact that developer tools were loaded on the VM instead of the host machine was a dramatic improvement for IT. At last, a software engineer’s machine was like everyone else's in the organization.

In addition to developer workstations, the integration and test environments were also virtualized. This allowed the release engineers to revert environments to their base snapshots in preparation for new releases. When we needed to branch testing, we would spool up another virtual environment. Our only limitation was the hardware on which those environments were deployed.

WHERE ARE WE GOING?

The last several years have seen virtual environments that exist beyond the firewall. The current front runners are Google App Engine, Amazon Web Services (AWS), Force.com by SalesForce and Microsoft’s Azure platform.

Consulting companies like mine that have an environment per client, and sometimes per client application, use up space on the network very quickly. Using cloud providers offers relief to the internal pressures related to providing developers with environments, storing them in various states of running, upgrading VM technologies, and expanding the network for the ever growing number of VMs.

Software Engineering teams interested in using cloud providers need to decide which flavor of Internet-based cloud infrastructure is appropriate for them; extending a developers workspace and “being” the developers workspace.

Each of the big four allow developers to use existing IDEs to extend their development environment. Here are the major IDEs and the providers support.

• Visual Studio: Azure
• Eclipse: Google App Engine, AWS, Force.com, Azure
• NetBeans: Google App Engine, AWS

Each of the big four support one or more programming languages. This list below is not comprehensive but provides the major languages:

• Azure: C#, Java, PHP, Python
• Google App Engine: Java, Python
• AWS: C++, C#, Java, Python
• Force.com: Active Script, Apex, Visual Force

As you can see above, which platform you choose is dependent on your internal expertise. Force.com is the only one that does not provide support for the major languages. However, after using it myself, it’s a more rapid development environment – somewhat like a 4 GL – than the others and will be a player for the long haul.

The appropriate infrastructure associated with the decision to move to a cloud provider is a decision to think through carefully. The various configurations are virtually (pun intended) unlimited, e.g. have the entirety of your virtual environments either behind or in front of the firewall, split environments between behind and in front of the firewall, and deciding which, if any, segments of the infrastructure are public.

The adoption of cloud providers is a non-trivial decision as is the design of the appropriate infrastructure. However, as a manager, software engineer, and consultant, the cloud is likely to be a catalyst for a trend that offers developers an opportunity to think more about development and less about infrastructure. Which is as it should be.

Will Google Wave Re-Define the Blog Paradigm?

As we all know, blogs are a social networking tool to desseminate information and capture feedback. The tools currently in use almost provide the mechanism for discussion but don't quite get there. Google Wave holds promise of bringing blogs to the conversational promiseland - at least in the world of typed messages.

Currently, the process works like this: Someone writes a blog article that is typically several paragraphs long with several points linked together by logical reasoning. After visitors read the article, they input a comment at the bottom. The comments usually consist of general commentary on the article subject matter, specific commentary on one or more points within the article, or extending the article through logical relationships to subject matter not discussed in the article.

Commentary from a blogs reader ship currently consists of narrative pointers into the article, e.g. "I disagree with you when you state...". These narrative pointers become exponentially more cumbersome with high volumes of visitors and comments. Discussion is lost because it quickly becomes impossible to follow the massive array of narrative pointers contained in the set of disparate comments.

Google Wave blows up this paradigm by allowing inline commentary within the article. Comments are no longer aggregated by visitor containing multiple points at multiple locations within the article. Comments become decentralized and are input directly in the article at the location where the specific points are made. This allows follow-on visitors to add to the discussion on just those points with which they have interest and absolved of the prerequisite filtering of content and comments that currently exists. The playback feature is also a pretty cool way of being able to read the article without comments and follow visitor feedback sequentially as they occurred.

I posted much of the text from this blog to a wave that I made public. Virtually immediately after posting it, I started getting inline feedback from a Wave user. If you have a Google Wave account you can view it by searching for with:public google notarangelo. There's some interesting commentary.

If you want to contact me using this new and interesting medium - and what I think is the future of blogging - then get a Google Wave account (I have some invites available if you need one) and ping me jack.notarangelo@googlewave.com.

You can read more about Google wave at http://wave.google.com/.

Economic Darwinism

Over the summer, I wrote an article in this space called It's Winter in July. The premise being that in these tough economic times everything you've done to this point in your career has culminated to your current market value. This article extends that subject in the form of what I call Economic Darwinism. Surprisingly, my Google searches on the term came up with hits but not in the way that I think of it.

Economic Darwinism in my mind is related to survival of the fittest. We have been in this downturn for over a year and it's been difficult. Lot's of layoffs, attrition, hard looks at the way we do business, and in many cases lots of change.

For the past year or so you have been in survival mode which is a good and healthy process. If you have survived being in survival mode then it likely means that you are built for what you are doing.

When you look around at your respective team members you likely see a strong set of individuals that as a group can deliver whatever needs to be delivered. In a culture of meritocracy, which for the most part describes software engineering, we have been transformed into lean teams with a kick-ass set of players that gets shit done. Yes, it was painful getting to this point, but the result is good.

If that accurately describes your situation, then from a leadership perspective the view should be "The recovery starts with us. The inevitable new phase of growth will be through this team. And through that inevitable growth there are opportunities for everyone."

Morale, whether poor or euphoric, is a state of mind. The thoughts bouncing around your cranium, regardless of whether they are verbalized or not, influence those whom you lead. It's your belief system that will - and does - significantly influence whether your team feels confident or insecure. Look around, see the strength of your team, and know that the recovery starts with you. If you believe that, confidence/morale will increase. If you choose otherwise then you should expect the malaise to continue.

Let there be no doubt that which ever side you land is all in your head. Your harvest is in part dependent on the seeds sown now in the Spring of this recovery.

Cloud Computing: Hype vs. Reality

The term cloud computing is a recent branding effort for an umbrella set of hosted service offerings. Of course hosted offerings have been around for decades - SalesForce.com has been providing an Internet-based (i.e. cloud computing) CRM solution since 2000. There are also a plethora of other vendors with offerings related to Platform as a Service (PaaS), Infrastructure as a Service (IaaS), and Software as a Service (SaaS).

Cloud configurations fall into three categories – Public, Hybrid, and Private.

Public: Public cloud offerings are those that are typically subscription-based where all of the hardware and software purchases and maintenance are abstracted away from the customer.

Hybrid: Hybrid cloud offerings are provided with a combination of abstracted hardware and software with private infrastructure configurations. It is presumed that this will be the most likely configuration as cloud services become more mature and mainstream.

Private: A private cloud is an Internet-based offering where the entire infrastructure is managed by customer personnel and typically has security applied to limit usage to authorized personnel.

There are some who will refute my definitions above as too limited because they consider on-site virtualized environments as a valid cloud configuration. In my view, if the service isn't Internet-based then it's not in the cloud. Time will be the arbiter of this amorphous aspect of cloud computing.

So why has something that has been around for so long now become worthy of re-branding? The major drivers are the ubiquity of the Internet, the reliability of Internet connections, the maturity of web-based interfacing technologies, and the rapidly expanding blur between desktop and web.

The remaining obstacle for most organizations with moving to the cloud model is security.
Moving sensitive data and intellectual property to a cloud provider is a risk to be considered carefully. An internal infrastructure can be completely isolated from the outside world where malicious activity is voracious to say the least. In fact, it's safe to say that because of intellectual property and the sensitive nature of the data being stored, that for some companies there will always be a need for an internal infrastructure. Government regulations regarding privacy and organizations obligations to ensure protection of personal data will continue to be a driving force for internal resources.

There is also the counterpoint that because of perception and the fact that their services are in the cloud, cloud vendors focus significantly more attention to security than many internal IT teams in organizations where IT is not the core service. That is a valid point that deserves consideration.

You could also make a case that cloud computing implemented as the status quo across all industries and organization size - in its current form – could be considered a national economic risk. As cloud computing proliferates and third parties develop widgets of functionality as a subscription-based model, resources from disparate cloud environments will become interwoven into applications resulting in nested dependencies. When developing the next killer application, why invent a wheel when you can subscribe to one and integrate it into its processes? A benign or malicious act that triggers a negative event across the Internet could have significant economic consequences depending on its impact. There could also be inherent compatibility issues when cobbling unrelated cloud-based widgets together.

Cloud computing has also been touted as the return to dumb terminals with all software being cloud and subscription-based. I can not see that happening in the short to mid-term. Much of this article was written on my vacation without wireless Internet access. My writing was done offline using software local to my computer. Installing binaries locally is going to be a way of life for the foreseeable future albeit possibly a bit neutered in some instances since more and more of what appears to be local functionality in many applications is actually sourced from the web. (e.g. online help, lists of templates, installing add-ins)

If you are reading this because you are interested in cloud computing and are considering how to leverage it for your organization then you are ahead of the curve. Take your time, develop a long-term strategy (including governance policies), implement in controlled phases, and seek advice from professionals to ensure your implementation is the right one for your organization.