Test automation advice for distributed teams

While traveling around the USA, I’ve learned a lot about working remotely — on the go.

I signed up with a service called Clarity to offer advice to startups on setting up test automation & continuous integration for distributed teams.

For a limited time, I’ll be offering free 15 minute sessions, but I’ll also be donating all my proceeds to OSSO – the orphanage support services organization that my wife volunteered at in Ecuador.


On the road at last

Today we loaded up all our things into an RV (and a 10×20 storage unit) and drove away from our home in Orem, Utah to Winnemucca, Nevada.

15 - 1

It’s a lot farther than you think.  A lot more preparation goes into planning a move like this than it sounds like.  But a lot less than you’d think.  And probably we should have done a lot more than we did.

It’s only the first stop on our year long (or more) plan to travel around the country as a family and see what we can see.  While I work remotely and Kelsey home schools our kids.

Our first destination is Oregon.  To see Crater Lake, and show my family where I used to live.  Then to Selenium Conf in Portland, and then Seattle, and then a meandering line from there to Florida where I have another work conference in December.

That’s a lot of miles — and it doesn’t leave a lot of time for stopping to smell the roses.

As the day of our departure neared, I didn’t think we could do it.  While I worked, Kelsey cleaned the house and put into storage (or sold) all that we had.

I was supposed to fix up the RV.

I ordered & installed solar panels, a new inverter and batteries capable of running my office on wheels.  But I didn’t get it hooked up to the house power.  So we have 4 100 watt flexible solar panels mounted on the roof with 3M Dura Lock (think super velcro) generating power to loose wires, a charge controller (Midnite Solar Kid) in the drawer, and a 2000 watt Samlex inverter & 12 volt, 150 amp hour batteries mounted in a storage compartment and not hooked up to either the solar panels or the house electrical system.

I was also supposed to install our tow package.  It didn’t come until last weekend, and I didn’t get it installed either.  But Kelsey got the house done and we put our last load in storage this morning and checked out with the landlord a bit late.  By the time we left, it was 11:30am (not nine).

Because the tow package wasn’t hooked up, I drove the truck behind Kelsey in the RV with the kids divided between us an switching back and forth at every stop — of which there were many.  8 hours into a 5 hour trip, I calculated we were averaging 35 miles per hour.  Finally we pulled into an RV park in Winnemucca, and we made our destination just before dark at 8pm.

Not only was it too late, but there was no pool.  And it meant missing a full day of work (plus a half day yesterday where I helped finish cleaning & packing.)

Everyone was in meltdown mode, except baby Mikey who was happy to be out of his car seat and went to bed quickly.  Kelsey is frazzled from driving the RV for 9 1/2 hours — which is much more demanding than driving a regular car.

But now everyone is sleeping happily, and hopefully ready to continue our journey.

I’d hoped to drive to an Opal Mine in the morning and work there tomorrow, but we’ll stay in Winnemucca and take it easy, and then maybe put some miles on in the night.  Rather than making travel easier, an RV seems to just make it slower

I want to get to our destination in Oregon by Sunday morning so I can go to church and see if I can meet anyone I knew back when I was in school.

We still have piles of stuff all over that we need to figure out how to stash or learn how to let go of.  And we still need to build bunks for the kids.  But we made it.  It’s less about having the right plan and more about committing to do something and then do it.

Using pyenv with virtualenv and pip cheat sheet

I made some notes on how to use a specific python version with pyenv and using virtualenv to manage packages.

using pyenv



$ git clone https://github.com/yyuu/pyenv.git ~/.pyenv


$ export PYENV_ROOT="$HOME/.pyenv"
$ export PATH="$PYENV_ROOT/bin:$PATH"
$ eval "$(pyenv init -)"

use command completion

$ source ~/.pyenv/completions/pyenv.bash

(homebrew on mac)

$ mkdir ~/pyenv/completions
$ cp /usr/local/Cellar/pyenv/20150601/completions/pyenv.bash ~/.pyenv/completions/pyenv.bash

use pyenv to install python versions

$ pyenv install 2.7.10
$ pyenv install 3.4.3

choose a python version with pyenv

$ pyenv shell PYENV_VERSION=2.7.10

upgrade pyenv

$ cd ~/.pyenv; git pull

(homebrew on mac)

$ brew upgrade pyenv

run `pyenv rehash` when you install a new python or binary package

$ pyenv rehash

install pyenv-pip-rehash to rehash automatically when you use pip

(as pyenv plugin)

$ git clone https://github.com/yyuu/pyenv-pip-rehash.git ~/.pyenv/plugins/pyenv-pip-rehash

(homebrew on mac)

$ brew install homebrew/boneyard/pyenv-pip-rehash

install pyenv-virtualenv

$ git clone https://github.com/yyuu/pyenv-virtualenv.git ~/.pyenv/plugins/pyenv-virtualenv

intialize pyenv-virtualenv

$ eval "$(pyenv virtualenv-init -)"

(homebrew on mac)

$ brew install pyenv-virtualenv

create a virtualenv with pyenv

$ pyenv virtualenv 2.7.10 my-virtual-env

list existing virtualenvs

$ pyenv virtualenvs

use a virtualenv

$ pyenv activate my-virtual-env
$ pyenv deactivate my-virtual-env

delete a virtualenv

$ pyenv uninstasll my-virtual-env

Testing SEO

I wrote the following post on LinkedIn after thinking about SEO and marketing for a friend’s business and how it relates to testing.

I’m not an SEO or marketing expert. But I am a testing expert.

And I am an entrepreneur who thinks about SEO and marketing. I create content about software testing (in part) to hep find customers for my test consulting business and network with other software testing and development professionals. I’ve also done my fair share of web development.

I’d like to know more about SEO, inbound marketing, and creating online content and connecting through social media. As a tester, I’m curious about the metrics and strategies SEO experts use to test their effectiveness and shape their content creation strategies accordingly.

So, I guess what I’m getting at is that I’d like to meet, and get to know and SEO expert and learn how they measure success and what their clients expect. I’d like to serve an SEO “apprenticeship” if you will, and maybe exchange services — I can build tools for content generation and distribution, measuring SEO effectiveness, and actions for turning views into actionable leads.

I’d also like to talk with businesses who are interested in SEO and inbound marketing, hearing what their strategies are, learning from their successes and failures, and maybe help them test the effectiveness of their SEO and marketing campaigns.

Testing facts – problems and solutions

Here are some facts about software testing I compiled to help me seek clarity about what is needed and what can be done to improve the state of software testing.

  1. Software testing is a multi-billion dollar industry.
  2. There are three segments to the software testing marketplace: tools, services, and infrastructure.
  3. Testing is a young and growing discipline.
  4. Testing is a craft.
    There is no strong academic or vocational discipline for testing.
    Some great preliminary work has been done to codify testing practices, but testing is primarily learned by mentorship and trial and error.
    There is no clear career path to testing.
    No one says “I’m going to be a tester when I grow up.”
    Indeed, few people know of the role before entering into it.
    Fewer still set out to become a tester.
  5. Testing consists of two primary facets: regression testing and exploratory testing.
    Both are necessary. The way they are approached should be different.
    The process should be different for regression testing and exploratory testing.
    A single tool does not fit well with both processes.
  6. There are two means to accomplish testing: manual testing and automation.
    There is a gap in skill sets between the two.
    There are some things that can only be accomplished well through manual testing.
    There are some things that can only be accomplished well through automation.
    There are some things that are better suited to either manual testing or automation.
    Few people understand the needs of both manual testing and automation.
    Fewer have the skill set and inclination to do both.
    There is a gap in tools between automation and manual testing.
  7. Testing is largely inefficient.
    Many testers are poorly trained.
    Many testers developing automation are inexperienced in development or lack formal development training.
    Software development best practices are rarely practiced in test automation — largely due to this lack of training.
    Testing tools are marketed to and geared towards test managers and gathering metrics, not tester productivity.
    Testing process is often not clearly defined.
    The features and requirements to test are often not clearly elucidated.
    Even when they are, they are not well prioritized.
  8. Testing is often an afterthought.
    Testing organizations are often understaffed and resources.
    Testers are often considered less valuable than developers.
    Developers often don’t want to test.
    Testing environments are often inadequate.
    Time available for testing is often cut short.
  9. Expert testers are rare.
    This may have to do with the lack of rewards and appreciation testing receives.
    There is not a clear senior career path — senior testers move into management, development, or business analysis.
    Senior testers should be mentoring junior testers.
    Senior testers should be designing testing tools and processes.
    Senior testers often make less than junior developers and low level managers.
    Many testers burn out.
    Including those who have a passion for testing.
    A passion for testing is rare.
    Many testers see testing as a stepping stone to development or IT management.
    It is difficult to break out of the testing “ghetto” once labeled a tester.
    Expert testers are good developers.
    Senior testers understand the business as well or better than any business analyst.
  10. Creating something (development) and testing something are two different skillsets.
    They have different goals.
    It requires a different perspective.
    Verifying something works and finding ways something will not work are not recipricol functions.
  11. Testers are focused on quality.
    No other part of the organization is as concerned with quality.
    Development strives for quality.
    Customer support feels the effects of poor quality.
    Testing measures quality.
    Quality is most often the item cut from the list: Quality, Speed, Price — pick two.
    Quality is rewarded in the marketplace more than quickness or cheapness.
    A quality perspective can be taught.
    Technical skills can be taught.
  12. Organizations need better testing tools and processes, and they need expert testers to use them.
  13. They need dedicated testing infrastructure, and they need someone who can help them see how to implement and organize testing.

I can help you:

  • Identify and implement tools and processes for testing, requirements & defect tracking.
  • Provision and configure IT infrastructure for test environments and tools.
  • Create a culture of quality and enable testers, developers, and business to work together to achieve this goal.
  • Find, interview, and train expert testers for your organization.
  • Provide additional testing resources to fill your need for scarce resources.
  • Develop valuable, maintainable test automation and train your testers how to do the same.
  • Bring the results of manual and automated testing (including developer tests) together for clarity on what business features are actually being tested.
  • Build products in a way that makes testing easier.
  • Inject testing into every step of your business process from initial concept to customer feedback.

I’m focused on building tools and defining processes that make testing easier and more effective. I’m also learning that testing skills and training are often lacking, and that good testers are often under-utilized, under-appreciated, and hindered by technical roadblocks, management apathy, and lack of organizational clarity about what testing provides and what testers need to accomplish their goals.

I want to improving testing, with these three goals:

1. Finding and developing expert testers.
2. Building better tools for testers.
3. Helping organizations improve quality processes.

Distributed teams vs Offshoring

I posted a reply to Phil Young on the MagenTys blog about “Re-Shoring.”  Below are my comments on that post.

I used to think that communication, culture, and face-to-face interaction were the important factors in why offshoring has failed.

But now I think it has more to do with the quality of services. Not that Indian or Chinese (or whatever) testers are inferior, but apart from the cultural and communication problems, and despite the domain knowledge gap, the problem is really that offshoring as a strategy is flawed.

The idea that providing warm bodies to fill chairs in a cheaper location is a solution to quality costs is erronious. You’d have the same quality control problems if you outsourced a key component of your business to Arkansas (or Wales).

The act of delegating tasks to someone whose priorities are not your own is the fatal flaw. They are incentivized to produce numbers at low cost. That’s the core problem — on top on any other problems inherent with remote teams in general.

But, remote teams — even distributed teams — can succeed if they are critical stakeholders in the business and not merely vendors. A distributed team can take advantage of the best talent wherever it is, not just in London (or Silicon Valley.)

Vendors, on the other hand, should provide a clear product or service, not be a part of your business strategy. And quality should be an important part of your business strategy.

Vendors are by nature a centralized entity with their own quality and business concerns. That’s not to say you can’t find a high quality “quality” vendor, but I don’t see how it works for the typical use case.

Of course, take my spiel with a grain of salt. I’m a tester who works remotely and a consultant who helps build distributed teams, so I may be biased, but I believe in what I’m selling.