I've been reading applications, and I've noticed a couple things.
First, a number of the letters of recommendation are targeted, at least by an address and salutation, specifically at our search. I've never noticed that before. It means letter writers are reworking letters (at least minimally) specifically for each institution the applicant is applying to. Maybe that's not many, but holy cow, it's a whole lot of work. And it really doesn't seem necessary or even really helpful. It's not like the faculty at Prestigious R1 are actually likely to know enough about a regional comprehensive such as NorthWoods U to really make the strategy effective. And it's got to make things MUCH more complicated for submitting letters on time. (Whose letter writers weren't always pushing the deadline? Anyone not have that issue? So it's got to add stress.)
The second thing is how few of the application letters clearly address some of the stuff in our ad. If we put in our ad that, say, speaking a foreign language is vital, then the letter should tell us about the foreign language the applicant speaks and how well.
In my role as future chair, I've talked to some folks who are on the market in different fields, and at least one has told me that doing an invited talk is a real CV booster. From my point of view, it's so not. A conference or three is great, but an invited talk doesn't really catch my eye. And given that a lot of candidates have multiple conference presentations, an invited talk basically disappears into the background. It's certainly not the make or break thing that my colleague seemed to think. For others?
What do folks see as make or break?
For me, a well written letter that addresses our job ad requirements. Research that sounds interesting.
A clear CV that shows growth and ... exploration, experience, interesting work.