Distributed teams vs Offshoring

I posted a reply to Phil Young on the MagenTys blog about “Re-Shoring.”  Below are my comments on that post.

I used to think that communication, culture, and face-to-face interaction were the important factors in why offshoring has failed.

But now I think it has more to do with the quality of services. Not that Indian or Chinese (or whatever) testers are inferior, but apart from the cultural and communication problems, and despite the domain knowledge gap, the problem is really that offshoring as a strategy is flawed.

The idea that providing warm bodies to fill chairs in a cheaper location is a solution to quality costs is erronious. You’d have the same quality control problems if you outsourced a key component of your business to Arkansas (or Wales).

The act of delegating tasks to someone whose priorities are not your own is the fatal flaw. They are incentivized to produce numbers at low cost. That’s the core problem — on top on any other problems inherent with remote teams in general.

But, remote teams — even distributed teams — can succeed if they are critical stakeholders in the business and not merely vendors. A distributed team can take advantage of the best talent wherever it is, not just in London (or Silicon Valley.)

Vendors, on the other hand, should provide a clear product or service, not be a part of your business strategy. And quality should be an important part of your business strategy.

Vendors are by nature a centralized entity with their own quality and business concerns. That’s not to say you can’t find a high quality “quality” vendor, but I don’t see how it works for the typical use case.

Of course, take my spiel with a grain of salt. I’m a tester who works remotely and a consultant who helps build distributed teams, so I may be biased, but I believe in what I’m selling.

Dynamic languages should be ashamed of themselves

It’s time for a rant on dynamically typed languages.

Frankly, I’m embarrassed by the state of scripting languages these days.

The Python, Ruby and Javascript communities have wandered so far from the principle of getting things done simply and quickly in a misguided attempt to cope with their insecurity of not being “big boy languages” or “hard enough to learn” that they have gone out of their way to make things esoterically stupid and obtuse in what can only be seen as a way to alienate beginning users.

I can’t recommend any of them to new programmers.  Each has had their moment in the sun, usually shortly after the previous reigning scripting language has gone off the deep end.

Perl has long gone the way of esotericism, and if you’re interested in asking yourself (over and over again) — what is an object really, it’s a good language to revisit from a purely academic standpoint — or if you’re stuck at Amazon or maintaining an old media web site.

PHP frameworks struggled to catch up with Ruby on Rails, but I defy anyone to start with Rails as their first framework nowadays.

I’ve never tried Laravel, but PHP has made strides for improvement but it’s probably too late, with too ugly of a codebase to move forward.

Ruby refuses to test on anything but a Mac and is no longer supported on Windows (or operable with MySQL or SQLite or MongoDB on that platform).  It’s pretty much a free for all on Linux.  And while gem, bundler, rake, capistrano, less/compass, sinatra, etc are all nice tools, they should not be required for hello world.  And unless talk like Yoda you do, read well it does not.

Python has always had the sore thumb of whitespace issues — suprise!  people still cut and paste code from the internet.  And suprise again!  it’s in plain text, not your favorite editor’s in memory document map.  Python has also had Ruby envy and a proliferation of tools.  Pip is almost as easy to use as easy_install and almost as well supported.  But watch out for Python 3, and beware of 32/64 bit compatibility across libraries in this text-based scripting language.

Javascript emerged as a serious contender with node.js and the V8 engine which finally gave serverside javascript the performance and stability it needed to take off.  Combined with the JQuery locator function and compatibility wrapper, you finally had something exciting.

But creating cool apps fast got boring faster.  Npm was a great package manager — only it only worked on the server.  So create a different standard on the browser — or two or three.  And use br^H mungify to convert!  But then, why not use grunt, yeoman, and bower to really abstract the word “require” which really just does a good job of reading files into the interpreter.

And port every one use tool from ruby with a different syntax.

Still, I really liked javascript.

But wait.  Javascript is flexible — you don’t have to program in a straightforward syntax — so why should you?  Make everything a callback, and pass functions to every function — even if you don’t have events or async code.

Scala — you could have been the next big thing.  Optional dynamic typing — but maybe Groovy proved how hard it is to be a scripting language on the JVM and still have performance.  So instead, you try as hard as you can to tell people you can only program Haskell in Scala.

Okay, I’m done ranting.  Time to get some work done.

 

What programming language should you use for automation

I was also asked what programming language you should use for automation.  Here’s is my response to that question.

Regarding the programming language you use for automation — I don’t think it matters.

A few points to consider:

1)  What language will the product developers around you be using?

You should consider using what they use because you can ask them for help and it might make for a more consistent base.

2)  What platform will you be deploying to?

This isn’t a big factor, but if you’re using Linux, probably don’t use C#, but if you’re a Microsoft shop, there’s nothing wrong with it.  It will also be easier to work with your existing tool chain (e.g. build tools)

I don’t think anyone needs told not to use Objective-C even if they’re an all Apple shop.

3) Do you value static or dynamic typing?

They both have pluses.  I like that I can build a framework that’s intuitive to use (especially in an IDE) when I use static typing, but I often get frustrated by the hoops I have to jump through (using enums and collections) to do so.

Sometimes it’s easier to just get things done in a dynamic language.  And it is still true that they’re easier to learn and the lack of a compile step can help speed things up.  But type safety is nice too.  I guess it depends on your own personal style.

Shifting from manual testing to automation

I got an email from a fellow tester recently asking what he could do to move from manual testing to automation. Here’s my (slightly edited) reply:

Shifting from manual testing to automation involves two main things I can think of from the start.

The first is obviously technical. You need to learn programming. How much depends on the tool you use. Some tools help hide the details of programming, but since you mention Selenium, I’ll assume that’s not what you’re talking about.

It doesn’t take much to learn the basics of programming. But it’s easy to miss some important principles and work harder than necessary creating less than optimal code if you don’t understand good programming principles. I won’t go into that in detail here other than to emphasize that the better programmer you are, the more reusable and maintainable your automation will be.

Apart from the technical aspect of programming, I think good testers already have many of the traits of good programmers — analytical mindset, precision in language, creative problem solving, and and attention to detail.

The second thing you need when moving from manual testing to automation is more subtle. And that’s shifting from an exploratory mindset to a defensive mindset. More plainly, you won’t be looking for problems with automation, you’ll be looking to prevent problems.

Your goal with automation should be to check that software is valid, not to find problems. As a manual tester, you’re rewarded for finding subtle problems — but as an automated tester, you’ll be limited to what you can anticipate with your automation, or you’ll tear yourself up trying to write predictive code.

The goal of automation isn’t to find new defects, it’s to verify expected behavior, and to stop regressions.

There are two points to keep in mind when you write automation.

1) Everything can’t be automated, nor should it.

Some things are too hard to automate reliably — or take too long to do so. Some things are easier and faster to do manually. And automation can’t notice anything wrong that it’s not specifically told to check for. Humans can.

2) Automation frees up manual testers to do exploratory testing and provide rapid feedback on things that are not easily automated.

Automation doesn’t replace manual testing, it enables it.

You should consider automating things that are repetitive, error prone, or to slow to do by hand. But you should also realize that not having a human check an area of functionality means that the person would not notice if other things go wrong (even simple things) in related areas.

Personally, I still like testing. And I like that my automation helps me to do better testing. I’ve found that when automation is not an expression of my testing (e.g. when I’m only writing automation, and not also doing exploratory testing) that not only does quality suffer on the product, but that my automation is less effective than it should be.

2014 in review

The WordPress.com stats helper monkeys prepared a 2014 annual report for this blog.

Here's an excerpt:

The Louvre Museum has 8.5 million visitors per year. This blog was viewed about 70,000 times in 2014. If it were an exhibit at the Louvre Museum, it would take about 3 days for that many people to see it.

Click here to see the complete report.

Continuous Testing (part 2)

In my last post [Continuous Testing part 1] I described what a typical build script inside a continuous integration tool looks like. And how they’re often a nightmare to maintain.

I didn’t even get to the point I wanted to talk about — which is that
something about continuous integration is broken.

Part of the problem is that people are trying to code in text boxes and doing setup and configuration through a GUI.

With discipline and practice, you can have everything within a tidy, modular build script so that all your CI server is just calling that (and nothing else)

Your job looks like :

/path/to/build_script.sh $OPTION1 $OPTION2 $OPTION3

but no more than 3 options!

It becomes nothing more than a glorified GUI on top of a cron job.

Which is all a CI server should be.

But was it JWZ or ESR who said something about every program attempts to expand until it can send email (and parse lisp?)

I think that Continuous Integration promises something more.

That’s what “Continous Delivery” tries to reach for too — and has gotten bogged down in the task of provisioning and managing build & deployment environments.

We have two tasks here that are really quite separate:

  1. Building a product.
  2. Deploying it to a working environment and running it.

But they’re both trying to accomplish the same goal:

Testing

You want to build a product continuously so you can test it frequently. Both unit tests, and system tests.

The “I” in Continuous Integration stands for “Integration Testing”

Likewise, you want to deploy a production continuously so you can test it frequently.

The “D” in Continuous Delivery stands for “Integration Testing” too.

Because to test something completely you need to deploy it to a production-like environment and run it through it’s paces there.

It just so happens that the environment you build a product on is also almost always an environment you can run it on. Developers know this because that’s how they know it works when they’re building it.

This is called a development environment. Your build system needs the runtime to build it but it needs a lot of other things.

Developers need javac to compile their code, but they also need java to run it. If they’re building web apps, they need a web server to deploy it to, a browser to connect to the web server, and maybe a database or two to get data from. They probably also need to mock out a service or call a development version of a service that doesn’t scale.

But a production environment looks nothing like a development environment.

And a testing environment is always a compromise between the two.

Since developers build developer tools, they build them with themselves in mind.

They’re thinking about unit testing, mocking services, and deploying to dev environments — preferably with as few steps between compile and deploy as possible.

That’s why we have CI servers that work the way they do. They were build by developers for developers.

But dev environments aren’t good for integration testing.  Ever heard a developer say “it works on my box?”

Similarly, we have Continuous Delivery servers that work the way they do because they were build by (devops) for sysadmins. They’re interested in getting the tools and processes for provisioning and deploying products in place — because that’s what they care about.

But as I said, the purpose of CI & CD is to facilitate testing. What if we had a Continuous Testing server that was designed (if not build) by testers, for testers?

Maybe it’s a piece that fits in between (or after) Continuous Build and Continuous Delivery. Or maybe it’s part of the same tool. But I think it’s the next step.

Continuous Testing

I’ve done a lot of setup and administration of continuous integration servers — cruise control (including variations cruisecontrol.rb and cruisecontrol.net), luntbuild, hudson, jenkins, bamboo, TFS, go. I have my favorites (and not so favorites.)

I’ve seen them used well and abused for continuous delivery and deployment as well.

Ironically, the ones that seem to work best seem to have lots of shell scripts written in text boxes. These are a nightmare to maintain, and impossible to recover from.

They often look like this:

  1. Set a bunch of environment variables in the CI tool
  2. Have a bunch of environment variables set (or overridden) within the job script
  3. Take a few more environment variables as user selected configuration parameters for the job
  4. Do some environment cleanup
  5. And then run the “build script” – a shell script that calls a python script that does a bunch more of the same stuff – and then eventually calls maven or rake
ENV_VAR1=$DEFAULT ENV_VAR2=$FRAMEWORK_DEFAULT /usr/local/bin/custom_shell build_script.sh -ex $OPTION1 $OPTION2

build_script.sh:

ENV_VAR3="DEFAULT"
ENV_VAR4=$OPTION1
ENV_VAR5=${OPTION2:-"DEFAULT"}</pre>
ENV_VAR5=`some_script.sh $ENV_VAR1 $ENV_VAR2`
export ENV_VAR3; export ENV_VAR4; export ENV_VAR5</pre>
/usr/bin/python cleanup_script.py $OPTION1 "hard_coded_value"</pre>
/usr/local/bin/custom_python build_script.py

One of the first things I do when starting on a new project is pull these scripts (and their associated environment variable settings) into version control.

And then I delete the jobs and never think about them again. I wish.

But I put them under version control because I don’t want to lose them.

And then I start refactoring.  And then I start trying to rebuild the functionality from scratch.  And then I congratulate the build engineer (who’s probably a developer trying to get work done on a completely different project) on his job security.