This is Part 2 of a 2 part series on hiring developers. Read Part 1, How to Hire Programmers, if you’re hiring for full time positions. Read this article if you’re outsourcing.
I recently wrote about how to hire programmers which focuses on hiring full time employees. This post is tailored to help you learn how to hire outsourced developers. You’ll see that the two methods share a lot of similarities except hiring a contractor is a bit less involved as they are not going to be joining your team full time.
A little background … I’ve hired almost 10 remote, outsourced developers who were all overseas using the method I explain below. I feel that while its not fool proof, its the best method I’ve encountered and I feel that it has the best ROI with the lowest time commitment. Over the years I’ve used contract employees for my business and I’ve been able to create this process as a by-product of my experiences. Sometimes it went well, other times it didn’t and sometimes it was an absolute disaster. After refining my hiring practices many times over I was able to create the process below that always returned top notch talent for outsourced developers. I hope the process below does the same you.
The Process
There are 4 simple steps and if you cut out the job posting step then you actually only have three steps to go through. The goal of this process is to save you time and money while maximizing the chance of getting the best candidate you can find. Please note, during this process you will probably need to spend anywhere from $50-$250 depending on the contractors that you put through this review process – more on that later. You will be posting a job, reviewing the candidates and selecting the top X (usually at least the top 10). At that point you’ll hire them to do a simple programming task. Once they complete it (if they complete it) you’ll be able to review the code and determine who has the best implementation and then you usually will hire that person. The steps are listed below.
4 Steps to Hiring a Good Outsourced Developer
- Post a Job Description a Job Site
- Prelim Review: Hire the top Candidates (minimum 10) for 1 Hour
- The Programming Task
- Keep the Top Candidate(s)
Step 1: Post a Job Description on a Job Site
Goal: To create an interesting job posting that will attract quality applicants
Note: I prefer to use oDesk.com as the site where I hire my contract programers so this entire article is baed around that site.
Log into oDesk.com and post a job. Make sure the title accurately explains what you want the person to do. Do not post things like “JavaScript Ninja” or “Rails Badass”. Titles such as “Senior Rails Developer” or “Wordpress Developer” seem to fare a bit better than abstract titles. The description of the job should describe what you need in detail. Do not leave this as a simple one liner such as “Write good code and deliver on time.” Inform the candidate what you need from them. If its updating an existing code base inform them of what they’re getting into. If you’re a business person who does not understand code, explain what you want the product to do or what your product already does and why you need help. Don’t spend a ton of time on this. Maybe 10-15 minutes, max.
When hiring in oDesk you can provide various options about the job. For each of these sections I tend to use the following settings:
- Skills Required
- This is self explanatory. If you have a Rails site, put “Ruby on Rails” and probably “HTML” and “JavaScript”, etc.
- If its another technology, put those technologies here.
- How would you like to pay
- Hourly when doing open ended development such as new feature work.
- Fixed when the exact scope is defined. Such as: Install WordPress and configure theme to look like X.
- Estimated workload
- This varies person to person, but I usually have them on for Part Time
- Desired Experience Level
- Always choose Intermediate or Expert. Choosing beginner will get you some shoddy work that will eventually cost you 2x as much to replace later.
- Marketplace Visibility
- Anyone can find this job (UNLESS … I’m hiring a specific person)
- Preferred Qualifications
- Freelancer Type: No preference
- Minimum Feedback Score: No Preference
- The reason for this is because there are TONS of people who are experts who have never completed a job on oDesk before. Also, sometimes people get a bad score from the only job they did … not because it was bad work but because the person managing the product was terrible at their job. Don’t count them out.
- Hours billed on ODesk: No preference
- Again, you could have someone who is an expert who just joined. Don’t count them out.
- Location
- This one can be tricky. I personally do not care where they are. But, this is up to you. I usually put No Preference. However, if you are testing the various locations of the world to find what works best for you then choose that location.
- English Level: 5 out of 5
- This is a requirement. A lot of contractors on ODesk will say their 4/5 but when you converse with them via email, IM or Skype/Google Hangout you’ll realize they’re actually more like a 2 or a 3. Being able to communicate effectively is key to success. Without it, you’re fighting an uphill battle.
- Screening Questions
- I personally don’t add these as programming jobs are quite technical and either you know how to program well or you don’t.
Now, post the job.
Wait about 48 hours and then move onto step 2. During this time you’ll start having people apply for the position.
Step 2: Prelim Review – Hire the Top Candidates (at least 10)
Goal: To hire the top candidates who seem qualified in order to take them to the next step.
You’re probably going to get upwards of 25-60 applicants for your job posting from candidates all over the world. The first thing you will need to do is trim the list down. Here’s how you do it.
- Cut anyone whose English (or whatever your chosen 5/5 language is) is terrible and very difficult to read. This is an immediate red flag. If you can’t communicate now, how do you expect to communicate later?
- Cut anyone who does not have the skills to do the work. e.g. – If you’re hiring for a Rails developer and the candidates does not list Rails as one of their skills cut them from the list. You want the highest quality you can afford.
- Cut anyone that doesn’t seem to fit the job description. e.g. – If you want someone to develop a Game for you, a line of business web developer probably isn’t the best fit for you.
- Cut anyone who is way out of your budget. Sure, that guy who’s charging $155 an hour looks great but maybe your budget is $25/hr. No point in entertaining the thought of getting a $155/hr candidate on the roster. You’ll waste their time and yours.
After 48-72 hours you’ll want to trim your list down to a minimum of 10 candidates.
You’re at the point where you are now going to spend real money. If the average wage of your applicants is between $5 and $25 and hour you’ll spend roughly $50-250 to find the right person (at most). You’ll want to hire all 10 candidates. Yes, I’m serious. However, when you hire them you will inform them that you’re going to hire them to complete a simple task that should take no longer than 1 hour. You will send them the link to the Programming Task (Step 3) so they can complete the task. This one hour is PAID to the contractor when they do the work. This shows good faith to the contractor so they do not feel like they’re having to do work for free. If you ask outsourcers/freelancers/contractors to do work for free you’ll lose ALL of the good/best developers immediately. When you pay someone a full hour of their rate they take you much more seriously.
Something interesting happens at this point in the process. An average of 50% of the candidates never continue past this step. The reason is not exactly known but I suspect it is because they simply do not know how to perform the task that is given to them. That is exactly what Step 3 is for, weeding out the candidates who can do the work from those who cannot.
Step 3: The Programming Task
End Goals: Identify if the candidate has the necessary ability to solve programming problems with code. Gain insight into the quality of code that is produced by this candidate.
You’ll notice that this step is very similar to Step 3 in the How to Hire A Programmer article with a few alterations sprinkled about. While the articles share similarities there are key differences in the process so please continue reading if you’ve already read the previous article on How to Hire a Programmer.
During the Programming Task* the candidate will be tasked with solving a real problem using the language that I specify. The coding task should not take an experienced programmer more than 1 hour to complete at most. I only pay for one hour of the contractors time. They can take as long as they want to get it done, but I’ll only pay for one hour. The goal is simple: To determine if the candidate can produce the code that is needed.
The programming challenge is the great equalizer. Not because the problem is necessarily difficult (it’s not, its easy), but because it allows you to gain insight into the quality of code that the person writes. It also answers a vast array of other questions as well:
- Does the candidate communicate effectively?
- Does the candidate know how to follow directions?
- Does the candidate know how to use Git and GitHub?
- Does the candidate follow coding best practices?
- Does the candidate know how to code?
- Does the candidate follow through and succeed?
I usually inform the candidate that the solution to the problem is due within 24 hours. The programming challenge is posted online on GitHub/BitBucket in a Git repo that is shared across all candidates. The same repo is used for everyone. Yes, that means that others can see the same answers as other applicants (via pull request). Thats fine because if someone wants to cheat it is easy to detect. If 3 people’s repo’s look exactly the same then the person who checked in the code first is most likely the person who wrote it. No two people write the exact same code every single time.
The repo contains all the necessary information about the task. The candidate is to follow the instructions and email me when their done. Here are a few example of some of the programming challenges I’ve created:
Feel free to fork these and save them for yourself. If you visited any of the sites above you’ll see that each of these examples had the candidate perform a series of steps. If the candidate is not sure of what to do, they could ask me for clarification or simply Google for the answer. When the candidate is done they’re to submit a pull request to me so I can review it.
This challenge is genius because it answers so many questions about the candidate. Namely, can they code and get something done on time. It also gives me the chance to review the code to see if it is a complete hack or if its something that is quite good. What I’ve found is that on average over 60% of candidates will never finish the problem – they simply don’t know how. Of the 40% that finish only about 20% actual do it correctly and usually at that point you’re left with either 1 or 2 candidates that are looking good. If you have more to choose from then thats a good problem to have!
It is also important to note that ODesk requires that the candidates install monitoring software that takes periodic screen shots of the work they’re doing so that you can review the work. This allows me to view their progress and semi-evaluate their process of solving the problem.
A common concern when letting someone do this task is: What happens if they cheat? What if they call their friend or have someone else do it for them? That is a possibility, but I see this as a real world problem. Sometimes programmers don’t know how to solve a problem so they have to call for help. They call friends, IM them, look stuff up on the internet or just find an open source project that does what they want and they take the code and modify it. Thats the world programmers live in. If I do that at an office or at home, who cares. If you’re truly a hack I’ll know fairly quickly. If you do get hired, I’m going to sniff out a problem very quickly and at that point you’ll need to make the decision to keep the person or not. If you’re a business person who can’t sniff this out, you may notice other things slipping such as timeframes, budgets etc. Any red flags are exactly that, red flags – so evaluate each one with caution.
If the candidate fails to complete the programming challenge, thats an immediate red flag. The task should be easy enough for an experienced programmer to finish in an hour. If the candidate cannot finish within the time allotted either the programming task is way too hard or the candidate is not suited for the position. If you’r using my Git repos from the links above (or something very similar) then this should not be the case. At this point I advise to stop the interview process for those who cannot complete the task and move onto the next candidate.
For those that do solve the problem correctly I like to review the code and if everything is satisfactory I’ll put them on the list of potential hires.
If you’re someone who cannot code, but you want to follow this process I highly suggest you find a friend, friend of the family, or coworker who is a programmer and ask them to help you. Offer to pay for 2 hours of their time so you can get their opinion of the candidates. Explain to them the process you’re going through so they know what their role is – a code reviewer. If you use this approach you will want to give the code reviewer instructions to give you a scale of “No/Maybe/Yes” for each code review. No means No – do not hire. Maybe means – its ok, not perfect, but workable. Yes means Yes, hire this person, this is good work. I include maybe because sometimes code looks good, but its not perfect. Thats OK. Sometimes you need to get out an MVP of a product out the door ASAP and it does not have to be perfect, it just has to work. The “Maybe” candidates are suitable for this because you’re testing a market/etc and you’re trying to get things done as fast as possible. However, you want to aim for YES candidates at all times if possible.
If you don’t find anyone who can solve the problem, repost the job posting to get some new leads. I’ve had to post a job a couple of times to get the correct person for the job. Yes, this costs more money, but it is much cheaper to pay this up front than months down the road.
Step 4: Keep the Top Candidate(s)
Goal: To hire the proper person/people for the position
In the previous step you identified who you can hire through code review and through initial filtering based upon communication through the application process. At this point you should be ready to hire your contractor. You may have had 2-3 people make it this far.
If you’re hiring for one position, hire the top candidate and let the other two know that you decided to go with someone else but you may be in contact with them shortly because you were impressed and you may need to grow your team. This leaves the door open for future conversation with those candidates. If you need to hire two people, hire the top two. You get the point here.
If you don’t end up with enough contractors, then repost the position.
Wrap Up
Hiring a contractor can be a challenging task, but if you follow the steps outlined above you’ll find that you can lock in a good outsourced developer and eliminate the cruft in a few easy steps. I hope this helps you land a great outsourced developer.
* I’d love to take full credit for creating the programming challenge on my own, but I cannot. The implementation of this challenge was inspired by a company by the name of Integrum (who no longer does programming) for this method of programming challenge.
If this process was in widespread use, I’m guessing there would be a lot of shady devs optimizing their resumes to win 1-hour contracts ^_^
This article itself does not even earn a 5/5 English Level rating.
The 1 hour contract is to ensure the person I’m about to hire does the quality work that they say they can do. If they complete that task successfully and I feel the code is sufficient then they will be hired. This would be on going work. Simply optimizing your resume for 1 hour contract wouldn’t guarantee a hire because the goal is to evaluate real work done by the candidate.
This is a great article on hiring!
I really appreciate you writing this. Thanks.
link to github api is wrong. it should be developer.github.com not deveop.github.com
Thanks for the catch. Fixed.
Nice insights. Thank you for sharing.
This is simple. If you are hiring an engineer you should be one too. This way you can hire faster and fire faster. No bullshit.
Outsourcing work overseas is 5x cheaper than local try-and-see-how-it-goes. Not a long term solution but a short term one to get stuff done. Hire fast, fire fast. So if you make a mistake move fast to get another one on the job. You wont bleed as much money from a mistake of outsourcing to the wrong candidate.
I have been through this a number of times over the years and documented my learnings on outsourcing pros & cons here: http://www.theroadtosiliconvalley.com/technology/outsourcing-work-overseas/
Thanks for this article. I’ll follow your process this weekend to hire an iOS developer to help finish my code. BTW, this post made the front page of HN! Congratulations!
Thanks Lou! I hope all is well with you! Great to hear from you. 🙂
This is a good article but I think you are not considering an important aspect and is the previous projects and the “footprint” of the candidates.
If I got an answer of some candidate and he (or she) gives me a github account and I can see he is the author of things like:
https://github.com/cloudinary/cloudinary_gem/blob/master/lib/cloudinary/api.rb
And I found he es member of Rails-Croatia group and he answered several questions to the newbies and he has a blog explaining the technical differences between Puma and Unicorn the “one hour programming task” will be useful only to know his communication skills.
“tram” should be corrected to “team”
“If you’re hiring for one position, hire the top candidate and let the other two know that you decided to go with someone else but you may be in contact with them shortly because you were impressed and you may need to grow your tram.”. Grow your tram. 🙂
Great article. I learned a lot. I am thinking about taking your Android developer challenge. I think it shows a lot about their comfort with understanding instructions, knowledge about GIT, and quality of their code. Thanks!
Fixed. Thanks!
Just fixed it. Thank you.
I am a very decent developer and sometimes work on oDesk. If you try to make me use the piece of shit oDesk screen tracker, you will lose me immediately.
The bottom line is, you either trust me enough to judge my work by the results or you don’t. If you don’t, making me install spyware software is just going to alienate me further. You’re the remote equivalent of the boss who drops by my desk every 15 mins that everybody hates.
I think you and the person who wrote this article don’t know the difference between git and github. Also, expecting someone to learn an arbitrary API (or to already know the github API) is silly.
No decent iOS developer will put up with this kind of nonsense. I say this as an accomplished iOS developer. But then, decent iOS developers charge $155 an hour, not $20 an hour.
Except that outsourcing overseas takes 7X longer, and thus is not a savings. You can get people who will follow these tasks, and can make a half assed android app that will show github issues.
But if you need to do an actual piece of software that is not trivial, you will find that overseas people simply cannot (or will not- I think they tend to work for dozens of companies at once) spend the time to do it.
This is a process that will find you coders, e.g.. PSD to HTML types. You will not find any “Developers” or “programmers” or “engineers” with this methodology.
First off, you’re not testing programming ability here, you’re testing to see if someone has already written something that uses the github API. (EG: if you think writing an android app that uses the github api is only an hours work you are underestimating the amount of work.)
Secondly, engineers/programmers/developers are people who create original work that solve problems that are particular to the needs of the product they are creating. You’re not asking them to do this, in fact you’re not asking for any programming, just coding.
I guess on odesk coders are what you will likely find anyway… but let me assure you, you’re not going to be finding any good programmers with this method.
Ah, I get it, you’re not a programmer. That’s the problem. Non-programmers telling other non-programmers how to hire programmers.
Blind leading blind.
I mainly use the screen sharing software during the interview and review process. Once I trust the developer to do their work (usually happens quite quickly) I never check the log again. Most of my developers stop using the tool and enter their time manually after that. You’re right, its all about trust. I simply don’t have time to be a micromanager, nor do I want to. Again, I use it as a review tool to help make decisions around who to hire.
Good tips! I love how your node and android programming challenges are essentially screenscraper apps… that was actually my first paid gig!
“Something interesting happens at this point in the process. An average
of 50% of the candidates never continue past this step. The reason is
not exactly known but I suspect it is because they simply do not know
how to perform the task that is given to them.”
Or they don’t like being insulted. I’ve written software for Apple, Sony, and Hitachi, including for PlayStation @ Sony. I think I’m good enough for you without your little “test”, which was probably created by a non-programmer. Also 1 hour isn’t enough time to do much of anything program wise (beyond “hello world”) and it’s sure not a good gauge of a person’s skills.
Real talent doesn’t like being insulted.
Next time I need brain surgery I’ll ask the surgeon to take a test for me, or to cut up some mice so I can see how he does.
Yeah, try that with good surgeons and see how many will agree to operate on you.
The only real way to judge developers effectively is to see their track record and what they’ve actually delivered, not quiz them like they’re on a game show.
Great developers are seriously talented people. Treat good talent that way and they’ll think YOU’RE the joke and walk instantly. It’s not because they can’t do the task.
Were you instrumental to the success of PlayStation at Sony? What’s that, you weren’t? Well I was. Then just who do you think you are quizzing me?
Maybe I should be the one giving YOU the quiz to see if you even know what you’re doing.
Have you ever written code? What products have you written? If not how are you qualified to eval potential developers?
“A lot of contractors on ODesk will say their 4/5 but when you converse with them via email, IM or Skype/Google Hangout you’ll realize they’re actually more like a 2 or a 3.”
If you’re going to say you want a 5/5 in English, you likely shouldn’t misspell “they’re”. Very amateur…
I came up with the same process for a task I wanted outsourced. I had 0 developers acceptably solve the initial project. Maybe I was just unlucky? Or maybe it’s because the domain is hard core Linux C++, which might have a smaller candidate pool?
1 hour to write “hello world” ? Damn, you *are* good.
The point here is that he wants someone on low cost. Real good programmers with actual good record with “heavy” names in industry would require far more than the maximum limit of 25$/h that Donn Felker said. Thus his target is good programmers with no previous heavy portfolio, that would work within low budget, and if I take into account the first rule, the good knowledge of English, I assume that most candidates are far away from USA with other mother tongue. So, his tests is not an insulting procedure, if you take into account which type of candidates actually he aims.
Uhm…:
they’re
contraction of
1. they are
Perhaps he’s referring to people who’d consider aiming for as many 1 hour contracts as possible, without actually putting any effort into serious attempts at being hired.
Imagine someone with a CV so convincing that numerous employers offer this person a 1 hour contract as a test. This person would then proceed to cash in on this 1 hour fee, without actually producing any useful work; they would only need to optimize for as many 1 hour contract chances as possible.
People begin to suspect foul play and write about this person? No problem, just ditch the account, create another fake online persona and proceed to gather a new round of 1 hour contracts.
Soon, as the number of fake candidates increases, you’ll find that it’s much more profitable in the long run to hire based on credentials and on a past history of successful projects, even if you’d pay more in hourly fees.
So you pay only $25/hr to a person who solves that challenge in 1 hour? (I checked NodeJS one). What is your success rate of hiring such a person at that low rate?
Nice article though!
Please check the usage of “its” in both articles. It should be “it’s” or “it is” in many cases I suppose.
Hello,
According to me it’s always makes sense to hire a mobile app developer
to create an app for your business, the question that usually arise that, how
does one find the right developer? It is never problematic to find mobile app
developers – it is only difficult to choose the right one which is able to
fulfill your needs. One more thing you have clearly mentioned the entire
process so effectively which is very good .I like your post it’s pretty much interesting.
iphone app development melbourne
Great tips Donn, I’ve just used your template to find a developer.
I’ve noticed that odesk now provides a ‘hire for trial project’ feature. When you message an applicant you can now select a trial project option. Very useful.
I’ll let you know how things work out, I’ve just hired 6 or 7 guys for a trial project.
You suggest that initial evaluation of developers may cost $50~$250. That looks like a market gap to fill: is there any paid-for service that objectively evaluates and rates developers’ proficiency (think an AngiesList for developers) that would be paid below $50 and saving, based on your figures, $200+ to the stakeholder.
Also, what is your best approach for hiring external developers for a multi-platform mobile app development since such scenario sets additional challenges for coding proficiency testing and team management? And a related question: what is, in your opinion, the most cost effective ADLM (Application Development Life-cycle Management) tool(s) to set in-house for hiring outsourced developers for a multi-platform mobile app development?
Guys, i am on the other side and looking for customers that would use our services.
We are an eastern Europe company of programmers/developers that operate mainly in the European markets and have many successful projects behind with good references. We are looking to expand to the US market as well. We work both – full projects end to end and also provide collocation of individual developers on demand. It seems to work very well assigning a team lead that is the interface between customer and the assigned developers. We are quite flexible in assignment of engineers, if those initially screened are not the right match we can replace them, although we know our team, all strong smart guys, so this would be rare case.
So this is a very brief summary of what we do and how we operate 🙂 If anyone is interested you can drop me an email on [email protected]
^^ This folks. This is how you hustle. Learn a thing or two.
I have had a few bad experiencing with outsourcing app development and decided to fix the underlying problem… communication!
I founded appR.co after a trip to India where I established an in person relationship with an amazing team. We do the design and product management here in states while they do the development. Doing so allows us to combine the creativity & design of the west with the technical skill & expertise of the east. The price benefits of outsourcing your work combined with the reliability and quality of work that you can only find here in the states.
And then if we believe in you and your idea we may also decide to invest in your company. Our mission is simple, help entrepreneurs succeed. Please do contact us and tell us how we can work together!
best article for hire mobile app and website developer for mobile app development and website development
This is really an informative blog for all those who want to hire outsourced developer.
The steps mentioned in the article are so interesting. After following these steps we found IndiaNIC Infotech Ltd… It is and emerging and very trusted IT company. It provides iphone application development, iOS app development, iPad app development and many more services.
If somebody wants to hire developers I suggest said company.
For more information: http://www.indianic.com
You wrote your own tests? Wouldn’t it be more effective to use online programming tests, such as these? They seem to be designed specifically for screening programmers.
Hello and thank you for such useful information!
However, you can outsource not only on oDesk, but also on other freelance boards. For example, there are a large number of qualified freelance developers here: https://www.xplace.com. They are willing to help with any project! And the website is commission free, so you won’t have to pay for outsourcing there.
Quick tip about hiring at oDesk (in fact now it’s Upwork).
So when you post your job, not just rely on the programmers who apply by themselves because 95% of them will be low-skilled guys from India and Pakistan. You need to make a search of candidates by yourself using the filters, like feedback, percent of successful work, fluent English etc. Not to spend a lot of time reviewing each developer you invite, it’s better to invite a lot, and then review after they agree.
This method will allow you significantly increase the quality of candidates you have.
Donn also suggests to get 10 most promising candidates and test them with 1 hour programming task. I would suggest little bit different way. You still can find top 10 programmers. But then just test them using coding tests to find top 3 developers. And then hire them for 2-3 hours programming task, because 1 hour is really too small to show how they write code.
Donn also mentions that most developers take $5-$25 per hour. I would never hire a guy who earn $5-$10, because the quality will be SO bad, so you will spend more money to override the code in future. I would say the minimum rate is $25. And I would better hire guys from East Europe rather than South East. The quality is much much higher.
You would assume so. Unfortunately cheating is rampant with these types of these. Online programming tests, standardized ones can be easily replicated. Creating a custom use case is key as they cannot simply find the answer online already done for them.
Great blog, you can have a look here , may be this can help you.
Recently I have read that it isn’t so difficult. Write them, just use the form on the site and start talking with 3–5 companies. Ask them about prices, deadlines, with whom they usually work. You understand who is the real professional by reading their business analyst answers. For ex, I’d recommend such companies as Axisbits, Rademade and Lemberg Solutions.