Posted by headalgorithm 19 hours ago
- Steve's company got acquihired by Amazon, granting him a free ticket in without all the torment of the multi-stage interview pipeline.
- It's a well known fact that it's easier to jump from one FAANG to another, so while interviewing at Google he had significant advantage, plus the blog gaining popularity.
- All of this has caused a deep down imposter syndrome, which resulted in an attempt to "improve" an interview process from the inside - but the wings were clipped pretty quickly by the corporate politics. It turns out that lawyers are not planning to reinvent anything there and hence are somewhat more important than engineers.
- The post itself is an self-applause over essentially a failed effort. "I've tried"
And even if you're interpretation is correct, you sound pretty rude. You're really gonna mock someone for not liking the interview process and trying to make it better?
So, he actually did go through the multi-stage interview pipeline.
Just because you can’t read well, you don’t have to be so cynical.
I know this author’s name but not his whole life story, but it reads like he got hired at Amazon before they acquired Geoworks and brought all of his former colleagues over.
That's some solid gaslighting right there! You're right Steve, I would rather be rejected by your team than wasting my time by doing something positive with my career.
"People would be pleased to have a rejection from us. They'd be proud to carry it sounds with them. Lucky them!"
It's funny, I see an article from Yegge and thought "I like that writer, I haven't read any of his stuff in a while, I'll see what he has to say." Then got to the end and see the links to gas town and gas city and remembered it was the same Yegge that while having accurate foresight about orchestration of agents also was a bit off the deep end in gas town.
But the biggest thing I see in this article is it really sounds like "here is the new company I landed at, and rather than make a post about its product, I'm going instead make a post about how terrible the problem it solves really is, and a post on a proposed solution. And the cues what I'll pop up in a few weeks and just coincidentally post about this new company that just happens to solve this problem in the way I've convinced everyone is the right solution."
While I don't have any evidence of this that's the feeling I left with. And if so, then "thought leaders" are a lot more interesting when not "talking their book."
However, a few companies do have a co-working day, where you work with various people on the team to solve a business problem. I wonder if he could have proposed that instead.
Ah of course, we have to do a traditional interview loop to evaluate 10 candidates before we can pick one. So you do the traditional interview loop, and then you have 6 months of provisional employment.
You haven't replaced anything, you've just added another level of hassle for everyone.
The false negatives suck for both candidate and hiring company who have accidentally rejected someone they would have like to have hired.
The benefit of the "provisional employment" process would be that even if it doesn't help avoid the false negatives, it would weed out the false positives so that the company at least ends up hiring people who are qualified and a good fit (even though due to the false negatives they may have rejected even better candidates).
Given how arbitrary in terms of talent mass layoffs are, there are of course tons of highly qualified out-of-work candidates (and due to age discrimination, maybe some of the best/most-experienced ones!), so this certainly might work to draw from that pool of candidates.
30/60/90 is pretty standard stuff. It’s literally the documentation and justification you need to fire the employee at the 90 day mark if you need to. Like a preemptive PIP if you will.
I don’t want to hire anyone who doesn’t want to be held accountable to what they say they can do anyway.
I'll talk to recruiters and interview at other companies on my spare time, as before.
What I feel like I'm reading here is someone who has been poisoned by FAANG hiring practices --- and they are terrible --- and has missed most of the work that's been done (outside of Google's admirable work in debunking their own processes).
I appreciate the "kitchen confidential" here, but with respect to Yegge, I think he's been working at the Olive Garden this whole time. Go stage at Gramercy Tavern! They're working at a different scale, yes, but you'll at least get a different perspective on the "gold standard".
As an interviewer, I much prefer the signals generated through a work-sample interview. I'm much more confident in the hiring recommendation than I get from a 1 hour zoom session. However, if I look at teams that were built through the work-sample and zoom interviews, I'm not sure the outcomes were that noticeably better.
As an interviewee, I think I understand the frustration being on the other side has. With an in-person interview, I often have a good sense that I bombed the interview or something to improve on or replay in my head, less surprising outcomes. On the work samples it's harder to know whether you're making mistakes, or are being out-competed by someone putting in 4 times the effort to polish the solution beyond what their regular work product would be. Although I had one really good outcome where the work-sample interview really flagged the internal dysfunction of a company.
And then with both interview processes, I still think there is a really big unknown on what the false no-hire rate is, how much effort is getting wasted rejecting candidates that would actually fit the team.
So having to choose a process as an interviewer, I'm with you and would always choose a work-sample interview. On whether it should be considered the "gold standard", I'm much more hesitant, I think there are some limitations that are still hard to control for.
I do wish Starfighter/Stockfighter model had gained more traction, would've been interesting to see a recruiting company specialize in this and then seeding the interview results to multiple companies model work out.
It's not just you. At the end of the day interviewing has been demonstrated to be close to a crapshoot in the best of circumstances, and very few interview schemes are the best of circumstances. Work samples are part of the optimal strategy [1] but even then the signal is quite low.
a) it discriminates against people who cannot spare 4+ hours of focused time on evenings/weekends to work on the problem. People with multiple jobs, single parents, etc.
b) in the age of AI it is no longer a reliable measure of someone's skill, for obvious reasons
Unlike Yegge, I haven't worked at FAANG, but the companies I have worked at all followed the same hiring practices and suffered from the same problems as he describes.
Provisional employment (or, if that's not possible, then well-paid internships) solve all of those issues. The candidate gets 3-6 months of stable employment, you as the employer get a large number of work-sample tests, and you can see how they use AI and how much.
Provisional employment doesn’t work for most cases, though. It might attract people who have no job and it might attract people who have so much saved that they are okay with potentially being let go after 90 days. But I imagine the vast majority of the potential employment pool are not willing to quit their current job to accept a “maybe” job from another company.
But the "way more than 4 hours" thing smuggles in a premise: that every candidate should be able to finish the challenge in the allotted time. But candidates with greater aptitude or conversance with the problem domain will complete work sample tests faster than candidates without, and selecting for those candidates is the point of hiring qualification.
If I ask you to write me a python function to convert OSGB easting/northing into WGS84 longitude/latitude the task has a very clearly defined scope. If you knock it out in a quarter of the allotted 4 hours, you've saved time. You can't use the remaining time to go further and demonstrate your mastery.
On the other hand if I ask you to write me a website for organising photos, there's no such thing as 'done' - no matter how good you are, after 4 hours you'll still be able to think of ways to make it faster, more beautiful, more featureful, more scalable, cheaper to operate, etc
Obviously, as a hiring manager I'll notice if you've spent 40 hours on the 4 hour task - but if you've spent 6 hours maybe I just think you're a fast worker with relevant experience and sharp tools. And my sense of how far you can get is calibrated by other prospective hires; if lots of people are spending 6 hours and claiming to have spent 4, my expectations will naturally be high.
This isn't a college application.
The rest of this I'm not interested in. For as long as we've been talking about hiring processes on HN, there have always been staunch defenders of interviews. Lots of people have spent time getting good at them, there are classes on it, there are books, there are drilling exercises. I don't anticipate talking those people out of their investment in interviewing.
For all its flaws, part of the benefit of an interview is it's time bound and equal for everyone. Similar to a test.
With AI coding this is also largely useless. These “build this thing in 4 hours” assignments come with a literal prepared prompt so that they can be churned out in 10 minutes.
Again, the underlying smuggled premise here is that candidates have to finish the work sample. No they don't. In fact, that's a strong sign it's not an effective work sample. A test that everybody passes isn't a real test; it's just a hazing ritual.
So you saved yourself/the team several possible hours of interviewing, and me quite a few hours - I think it took me about 10 to 15 minutes to see what you wanted in an engineer and that I was not it, and a total of 1 email which felt quite automated (whether it was or not) so there was a very low social cost as well.
> Again, the underlying smuggled premise here is that candidates have to finish the work sample. No they don't. In fact, that's a strong sign it's not an effective work sample. A test that everybody passes isn't a real test; it's just a hazing ritual.
Now this is an interesting take. Usually when people talk about these take home assignments, they talk about assessing the quality of the work. How good is the design? How is the coding? Is it efficient/elegant/whatever?
Here you take a much different approach, saying that the completion itself is the filter. If one person completes your assignment in the allotted 2 hours and another needs 12 but never tells you that, do you not care about that discrepancy?
I'm having a real hard time seeing how this isn't strictly better than an interview, which, as the article (and basically everything written in the last year about interviews) points out, is basically a random function.
If you are going to take a day off to do 3-4 in person interviews at a company then this slots in well.
The AI thing is an interesting problem, but a solvable one. We continue to hire resume-blind.
Would this company permit an employee taking 3 months unpaid leave to provisional hire somewhere else and have free choice whether they stay or go at the end of it.
Yes, standard interview loops also discriminate, and the more time they take, the more discriminatory they are. Any on-site requirements compound the issues.
Like Yegge says: provisional employment/internships solve all of these issues. You get the best of all worlds: stable employment for the candidate where they get paid a regular wage and aren't under a stressful interview setting, and lots and lots of work samples for you, the employer. It's not perfect. For example, it can be difficult to entrust the provisional employee/intern with anything impactful if you don't know whether they'll be employed at the end. But it is significantly better than the alternatives in most contexts.
It works. I’ve seen it in two different places.
At the second one, the fundamental realization I came to was that it is virtually no different than “regular” employment, where the new employee can get fired for not meeting expectations within an arbitrary time period after being hired. This can be months, or even weeks. From the perspective of the candidate, regular employment and provisional employment have roughly the same level of risk: in both cases they take a job where they might be let go at some point. The benefit of provisional employment is that they know how long they will be evaluated for and against whom. It turns out a lot of candidates do in fact like the all-cards-on-the-table approach and enjoy being given the opportunity to prove themselves on the job.
The conclusion at the end that bringing someone on board is the ideal method is true I'm sure, but even that runs into the issue that employee evaluation is an even worse situation than the interview process.
You can openly see some managers panic when they realize they have no idea what their employees have been doing for the last 6-12 months when they're asked to provide feedback.
1) The worst interview I ever had (BY FAR) was at Google--disrespectful people, no respect for time, I could go on and on. And I went back to try again to get that money showered on me. Worth it in the long run.
2) Their new system for "performance management" is a hoax. Just like at all other places, it "documents" what you should do so they can fire you more easily with unspoken rules and all sorts of arbitrary causes as well. A friend literally hit EVERY pre-agreed target and still got pushed out for "not delivering".
They told me it was all about pseudocode and how I think in advance, then on the actual interview they were being annoying about variable names and spaces after comma, while I was supposed to come up with some clever optimisation that boiled down to: "do you know this obscure theorem already? Cool you pass"
1. Standards got higher. Luckily if you got in early and proved yourself you are OK. But doesn't mean you would pass the current interview.
2. A marathon runner (with rare exceptions) can't run a marathon on a random day. They train for a specific date. Same with interview prep.
They did, but not with the intention of doing anything about the problem.
This is a question of reliability, the conceptual 'correlation' of a measurement instrument with itself when measuring the same thing.
Reliability is one of two major concepts in psychometrics, the other being validity, the conceptual correlation between a measurement instrument and that part of reality that you're hoping to measure.
The question behind validity is "I want to know X; if I measure Y, how helpful will that be?". And the question behind reliability is "if I measure Z, how accurate will that measurement be?"
https://en.wikipedia.org/wiki/Reliability_(statistics)
https://en.wikipedia.org/wiki/Construct_validity
Yegge calls out both concepts explicitly, though not by name, in this essay:
>> The outcomes from interviewing are statistically terrible. Google did wave upon wave of analysis over the years, and all the results were incredibly depressing.
>> [reliability] To name just a few off the top of my head: interviewers barely agreed with each other. Put the same candidate in front of two of our sharpest people and you’d routinely get a confident “strong hire” from one and a flat “no” from the other.
>> [validity, though the 'problem' here is strongly confounded by a restriction of range issue] And once people were actually on the job, their interview scores told you next to nothing about how they’d do
>> [reliability] Hell, some of our star performers failed their Google interviews four or five times, finally got in after 2+ years...
>> [validity] ...and then outshone everyone else.
The discussion of how interviewing outcomes are statistically terrible would benefit from naming the ways in which they're statistically terrible. Knowing the problem you have is an important step toward solving it.
(And as a side note, the last I heard from Google, you're not allowed to interview more often than once a year. Interviewing five times in two years would seem to violate that policy.)
It is a basic theorem that the validity of any instrument is bounded above by the square root of the reliability. It isn't possible for an unreliable instrument to be tightly correlated to reality, because it is, by definition, not tightly correlated with anything. That's what it means to be unreliable.
Thus, any company that wanted its hiring process to be good would necessarily be extremely concerned with making that process accurate; you need to come to the same decision when you assess the same person. This is something that interviews cannot achieve except at extreme cost. You'd need far more than five interviews to get a reliable assessment from them, despite the claim in this essay that "any more than four interviews and you're just playin' with your food". Of course, the Google interviews aren't supposed to be reliable anyway, so in that sense the claim is probably accurate.
The prescription Yegge offers is valid. Multi-month work assessments will give you a strong, reliable, and valid signal. They're also very expensive.
Another thing the essay completely glosses over is that this problem has been recognized for a long time, and we already know how to do assessments that are reliable, valid, and cheap to perform. They're called standardized tests.
The problem is that companies like Google that have evaluated their own hiring process, by comparing candidates "hiring score" with subsequent on-the-job performance, have found that there is little correlation. So, while the goal (be more concerned about false positives than false negatives) makes sense, their process of trying to achieve this is broken.
Perhaps we could administer this IQ test at age 12, so that the low-scoring individuals can go straight into the fast-food industry, and the rest can pick between the doctor/lawyer/SWE offers that will be showered upon them?
Hiring from a test that measures IQ is a very good idea (and there is a test that's commonly used for hiring purposes, the Wonderlic†); hiring from "an IQ test" is a bit less good. Anyone who wants to subvert the Raven's test will be able to do that. High-stakes tests need more security.
The concept of "IQ" can be toxic in contemporary American politics, so there are many more tests that "happen" to test IQ than there are tests that advertise themselves as testing IQ.
† https://psycnet.apa.org/record/1982-00123-001 : "correlations between Wonderlic IQs and WAIS Full Scale IQs were [0.93] for the main group and [0.91] for the cross-validation group". Note that this test involves only basic math and takes 12 minutes of the candidate's time.
From the candidate's POV what is mostly broken about the application process is all of the ATS gaming, resume tweaking, etc, just to avoid getting filtered out before an interview, as well as the inane leetcode screening that some companies are doing.
A better process might be to replace all initial profile/resume-based screening with task-based evaluation (evaluation could be at least semi-automated - if a computer/AI is going to reject me, I'd prefer it to be based on task evaluation job skills rather han ATS-filter avoidance skills!).
Lengthier on-site interviews/evaluations could also be task-based - for a developer role perhaps a 2-4hr peer-programming or problem solving task. Far less signal that a 3 month provisional hire of course, but maybe a better use of everyone's time than a traditional talk and brief whiteboard challenge process which is clearly failing as a useful filter.
I wouldn't expect companies to forgo the traditional touchy-feely team/culture fit type of screening as well, but better if this came after they'd already determined, as best they can, whether you've got the chops to actually do the job well.
The post touches on it (regulation about firing causing legal issues) but the unfortunate truth is that turning away false negatives has little cost to a business as long as you make sure to turn away false positives. Bad hires are extremely expensive, where as the lost revenue of turning down a good hire isn't nearly as bad (it's easy to try to hire them again later).
The stamp idea is good, reputation based hiring is important, as shown by the use of referrals in hiring. I've always wondered why we don't keep track of referrals inside companies in order to figure out who is recommending their peers who legitametly are good fits, vs those who are just trying to help out friends (who might not be the best hires).
Further I think there is also the problem that anti-corruption/anti-nepotism often harms hiring. It's much easier when working in a start up to hire the most talented people you know without other people sitting in, compared to at a big company where I know people who actively know good people they want to hire, but who can't be arsed with the long interview process and high chance of rejection.
If it wasn't for the regulation around firing some sort of staking your job on the line would be effective. You can refer someone and have them skip most of the process but your job is on the line if that was a bad call.
That's real useful work, if we don't hire them they can show it off to a different hiring company, and it makes the world a better place.
A man walks into a pet store. There's a parrot for $100, it says this parrot speaks perfect English . The one next to it is $1,000, and says, This parrot speaks 12 languages fluently.
Then there's a bedraggled looking, droopy, parrot, and its label simply says One Million Dollars.
Does it sing opera and has successfully run for President? the man asks with a sneer.
This parrot, says the store owner, _thinks_.
That's what this entire post is about - how to evaluate people with a series of attributes, score, correlate, blah blah blah.
Hire them, see if they think. If they don't, fire them. It's cheaper than this credential/signal rigmarole, most of which is about CYA legal b+llsh1t. Yes, it's a simplistic strategy and it doesn't work for Shoogle, Banthropic, Goober, whatever. You know what, boo f*cking hoo. You're a trillion dollar company, suck it up. You have a zombie horde at your doors and you're just upset the "true gems" are hard for you to spot amongst the slavering masses. You're going to heartlessly lay them off anyway in a few years. You SHOULD feel this pain and anguish of having to sort through them, constantly regretting all your choices. That's the only way to have balance in the Universe.
Edit: Didn’t link it initially because I thought it would be hard to find. Turns out it’s not. https://www.pushkin.fm/podcasts/revisionist-history/hamlet-w...
The fact that an organization cannot deal with such a case is a bigger problem in the first place. Eliminating incompetence and malice is among the basic skills of an organization.
Why is incompetence so discussed for ICs but rarely for management?
Maybe if more managers were competent and less conflict-averse they would do their jobs better and cycle out incompetence and bad intentions faster.
Ah wait, but middle managers specifically choose conflict-averse, easy to control, domesticated people. That's why you find this particular type of personality in management so often.
So the issue is incentives as well and bigger than initially thought.
And that's why the trades require you to perform years of work as an apprentice before you're ever qualified to pass an exam to become a journeyman. Not only is it like a years long internship, but you're paired with someone who has already passed a high bar. You're learning directly from vetted people, and those people can vouch for you.
> University of Waterloo famously sends their Computer Science students through a total of six internships, giving them roughly 2 years of real-world work experience before they graduate.
Still less time than an apprentice is required to work, but it's better than nothing. Most trades require you to be an apprentice for 4 to 5 years.
> The reason is, hiring engineers has historically been so competitive that you couldn’t convince a senior engineer to do an internship [...] So the industry converged on not requiring it.
Which is one reason the trades made it the law that you have to follow the apprenticeship in order to become a journeyman. "The contracting companies don't feel like hiring licensed workers" isn't an option. Companies don't do the right thing unless they're forced to.
This is why we need professional engineer certification for software engineers. We need a rigorous, time-tested, reliable process to ensure engineers have actually done the job in the right way. Otherwise it's a bad guessing game like Steve is explaining.
This is also why we need a software building code. We need to explicitly define exactly how you're supposed to engineer, so that we can create a certification that people can pass. Otherwise designing and building software is completely subjective. Engineering should not be subjective.
This is not some mysterious experimental idea that nobody knows if it'll work. Trades have been doing this for decades. It's not perfect, but it's much better than the alternative.