Enter the Matrix
Creating and Using a Decision Matrix
Last week’s two Shelf Lifes were all about how to find and vet a freelance editor to work on your manuscript. I mentioned in passing that this week I would share the decision-making tool I use when I’m deciding among multiple candidates—people, services, companies, or tools—that have different but equivalent merits.
That is to say, you may—in the course of evaluating freelance editors or, in fact, any pool of any type of providers for any job—find that no frontrunner appeared during your vetting process. Perhaps there are multiple candidates who all look good for the job, but no one who stands head and shoulders above the others or even ranks as first among equals. This is always a danger when you’re selecting candidates for anything based on multiple criteria. The candidate who stands out as best in Criteria A but was middling on Criteria B may be competing with someone who excels in Criteria B but is only average in Criteria A.
Anyway sometimes a list of candidates for something produces a clear frontrunner and sometimes it does not. Further, sometimes the choice of the “clear frontrunner” is not based in fact but in sneaky cognitive biases. For instance, anyone might find themself disposed more favorably to a candidate who:
Mirrors ourself in personal characteristics like race, gender, age group, and so on (the affinity bias).
Puts forth a more polished presentation, for instance, a fancier website than other candidates (the halo effect).
Is the first one we evaluate (the anchoring bias).
When you, as an individual, are hiring a freelancer out of your own pocket to work on your own manuscript, you can feel free to be as biased as you wish to be. You are not beholden to a higher authority to make sure you eliminate biases in your decision making. In my line of work I have to select candidates for employment and contracts and I need to demonstrate that I mitigated personal biases I may have in my selection process, so a decision-making process and matrix like those I will share below are helpful.
In any decision making process, there must be discrimination. This doesn’t mean discrimination based on characteristics like race or gender but rather discrimination in its dictionary sense, the process of distinguishing the differences and unique features of each. It’s impossible to make a sound decision without using the process of discrimination to find the differences between candidates to arrive at the best one for the job at hand. The trick is to eliminate or mitigate your personal biases to the extent possible so you can depend on the pertinent selection criteria to arrive at a decision.
Determine Your Criteria
The first step is figuring out the criteria on which you will rank your candidates. In last Thursday’s Shelf Life, How to Evaluate a Freelance Editor, I talked about vetting freelancers based on the triple constraint of cost, timeliness, and quality—which are three categories under which your criteria will fall. It’s not as simple as ranking candidates on cost, timeliness, and quality. If it were, this would be a very short Shelf Life indeed.
In addition to cost, timeliness, and quality, you may want to break the wide category of “quality” down into multiple categories. For instance, you might break “quality” into, for instance,
Experience—how much relevant editing experience does this editor have?
Fit—how close a fit is this editor’s experience for your particular manuscript?
Qualifications—outside of editing experience, what qualifications does this editor have that recommend them to work on your manuscript?
Testimonials/reviews—are other clients happy with this editor?
And so on. You may also wish to add an another category entirely—intangibles. Intangibles might include things like whether the freelancer offered extras like a meet-and-greet call or consultation, and how you felt about the freelancer after interacting with them via phone, email, or zoom.
The goal in mitigating subconscious biases is not to completely eliminate intangibles like “this candidate just had a good vibe.” Having good chemistry with your editor is a real factor in choosing a creative partner and not something you want to dismiss just because it isn’t quantifiable, measurable, or factual. Instead, you want to make sure you don’t give undue weight to a hunch, gut feeling, or personal liking for someone to the extent that you overlook other important criteria about that candidate and the others.
Once you have your categories and the accompanying criteria, put them into a grid. An Excel spreadsheet, a Word table, or even a hand-drawn grid in a notebook will work for this—but for additional helpful functionality, Excel is best (a downloadable template is available to download at the end of this article). Put your categories across the top at the head of each column.
This grid is the beginning of your decision matrix. Under each category, specify a few criteria that you will grade on a scale of one to five. Don’t go too crazy with the criteria or you’ll be at this forever. Limit it to the ones that are truly important to your decision. I limited myself in this case to three criteria per category. I have numbered each criteria under its respective category.
Weigh Your Criteria
Once you’ve chosen your criteria, you will assign a weight to each one, with a higher weight given to those criteria that are more important to you. You can do this by category or by criterion, depending on how granular you wish to get. For the purposes of today’s Shelf Life we’ll weigh by category. Let’s say that, for me, the most important factors are the cost and the prior experience. I want to get the most experienced editor I can for my money while sticking with my budget. Assuming I can’t budge on my budget at all, I’ll put cost as my most important factor. In this imaginary scenario my schedule is flexible and I’m willing to work with anyone as long as they are a good value for me based on cost and experience, so I will weigh timeliness and vibe lower. Fit will be weighted in the middle.
You can choose any weight scale you wish but since I’m working in Excel, I’ll go the easy route and weigh on a scale of zero to one, with zero meaning “this category holds no importance at all, why am I even ranking them in this?” and one meaning “this category is of the utmost importance to me.”
If all categories are equally important to you or if you want to simplify the process, weighting is very skippable. Consider this step totally optional.
Rank Each Candidate
Next, we’ll add our stable of candidates to the matrix and rank each one based on five categories with three criteria each for a total of 15 criteria. Keep in mind you can have any number of criteria and categories; more or fewer. There’s no need to put this much specificity into the decision, but you could also put in much more.
I’m using a one to five scale for ranking, with one being the lowest score and five the highest. Importantly, some of these rankings will be totally independent of each other and some do not have to be. Consider criterion 2 from the timeliness cargory—what does “a speedy turn time” mean? If I’m ranking three editors, I might give a 5 to the editor who promised the fastest turn time, a 1 to the editor who promised the slowest turn time, and a 3 to the editor whose turn time fell in the middle of the pack. However, I could also rank these independently. If, in my mind, “four weeks” is a speedy turn time I would award higher points to editors who promise a turn time of four weeks or fewer and lower points to editors who promise a turn time longer than four weeks.
A yes/no question might award either a 5 or a 1, with no possibility for an in-between score.
I’ve now ranked all my candidates based on the information I collected about and from them during the vetting process. Since I used Excel, the spreadsheet will now work some magic and give me my rankings.
Review Your Rankings and Make a Selection
If you are not weighting values, all you need do now is total each candidate’s row across to get their total score. In that case, in the example above, Candidate A is the clear winner with 60 points; Candidate B comes in second with 52 points, and Candidate C comes in third place with 42 points. After weighting their scores, the chips fall similarly with Candidate A coming in first and B in second, although the point spread is closer. Either way, Candidate A is the clear choice even though Candidate C gave the best vibes.
To arrive at the weighted scores, I multiplied the numbers in each candidates total category scores by the weight for that category and then summed them. So for Candidate A, in the above example, I multiply their Timeliness score total 13 by 0.2 (2.6), Cost score total 12 by 0.9 (10.8), Experience score total 12 by 0.8 (9.6), Fit score total 13 by 0.5 (6.5), and Vibe score total 10 by 0.3 (3) and sum those to get 32.5. I did that for Candidates B and C as well. I mean, so to speak. Excel did the math.
Next I’ll mix it up a bit. In the example below, Candidate C still has the fewest actual points with 42 while B has 52 and A has 55. However, Candidate C stuck the landing in cost and experience, my two highest-weighted categories, while A and B did well in other categories but poorly in cost and experience. In this example, although Candidate C had the fewest points overall, weighting kicked in and gave more priority to cost and experience, giving Candidate C the edge.
It is a nearer thing in the second example, though—the weighted total score spread is much closer. The weighting is doing its job, giving an edge to someone who excels in my most important categories but not to the exclusion of all else.
Depending on which scores are my real ones, I might solicit Candidate A or Candidate C. If my top choice declines, I can easily see how close my second and third choices are by reviewing their scores in this matrix and decide whether I will settle for my second choice or go back to the drawing board (for more information on the drawing board see last Tuesday’s Shelf Life, How to Find an Editor for Your Manuscript).
If you have questions that you'd like to see answered in Shelf Life, ideas for topics that you'd like to explore, or feedback on the newsletter, please feel free to contact me. I would love to hear from you.
For more information about who I am, what I do, and, most important, what my dog looks like, please visit my website.
After you have read a few posts, if you find that you're enjoying Shelf Life, please recommend it to your word-oriented friends.
What an excellent and helpful decision-making tool! Thank you so much for this!