The best interview process I've ever been a part of involved pair programming with the person for a couple hours, after doing the tech screening having a phone call with a member of the team. You never failed to know within a few minutes whether the person could do the job, and be a good coworker. This process worked so well, it created the best team, most productive team I've worked on in 20+ years in the industry, despite that company's other dysfunctions.
The problem with it is the same curse that has rotted so much of software culture—the need for a scalable process with high throughput. "We need to run through hundreds of candidates per position, not a half dozen, are you crazy? It doesn't matter if the net result is better, it's the metrics along the way that matter!"
I dislike pair programming interviews - as they currently exist - because they usually feel like a time-crunched exam. You don't realistically have the freedom to actually think as you would in actual pair programming. i.e. if you wag your tail chasing a bad end for 15 mins, this is a fail in an interview, but it's pretty realistic of real life work and entirely a non-problem. It's probably even good to test for at interview: how does a person work when they aren't working with an oracle that already knows the answer (ie: the interviewer)?
Pair programming with the person for a couple hours, maybe even on an actual feature, would probably work, assuming the candidate is compensated for their time. I can imagine it'd especially work for teams working on open source projects (Sentry, Zed, etc). Might not be as workable for companies whose work is entirely closed source.
Indeed, the other problem is what you mention: it doesn't scale to high throughput.
> i.e. if you wag your tail chasing a bad end for 15 mins, this is a fail in an interview
In all pair programming interviews I have run (which I will admit have been only a few) I would fail myself as an interviewer if I was not able to guide the interviewee away from a dead end within 15 minutes.
If the candidate wasn't able to understand the hints I was giving them, or just kept driving forward, then they would fail.
Exactly! Who calls "researching how to build X, but then letting my pair-programming partner fall down a rabbit hole so I can feel superior" "pair programming".
That's definitely up to the interviewer, in which a lot of discretion and trust has been placed. I think a lot of it also comes down to the culture of the company—whether they're cutthroat or supportive. As you get better people into the company, hopefully this improves over time. I know that when we did it, it was never about nailing it on the first try, it was literally about proving you knew how to program and were not an asshole. So, not the equivalent of reversing a binary tree on a whiteboard. The kinds of problems we worked on in the interviews weren't leetcode type problems, they were real tickets from our current project. Sometimes it was just doing stuff like making a new component or closing a bug, but those were the things we really did, so it felt like a better test.
Exactly. People go like “the ideal way to interview is <whatever they themselves are the best at>”. Pair programming interviews suck and don’t scale, just like every other alternate way of hiring.
IMO interviewing is the biggest bottleneck and if interviewing was decoupled from hiring then it wouldn't be a problem. But this requires a Guild-like organization to manage interviewing/vetting and for companies to use said Guild for hiring. The companies could then do a single team culture meeting (if they wanted) before hiring.
I wish I could remember who, but there is a company out there who conducts tech interviews to create a pool of candidates for their customers. Pretty sure there was a post here about it, but it's lost in my ocean of unread favorites.
Create a scalable, practical interview process where the result is reliably the best candidate, and you'll take over the world.
At this point in my career I no longer even believe that "best candidate" is a meaningful description except perhaps for highly specialized roles. We're all stumbling blindly in the dark.
Exactly. Its almost like optimising for finding your best possible match for marriage. You don't go over a billion prospects, you choose from the ones locally available to you, as they come.
it isn't about scale. It is about core principle of the tech hiring - all the companies hire only the best. Not only it is impossible to scale, it is plain impossible. Even if all the companies hired only "above the average" it would still be a pretty tall order :)
There is no one-size-fits-all for software developers. Different companies are looking to hire people at different experience/pay levels, with different skill sets, etc. Just as with dating, most companies are also going to have some understanding of their own "attractiveness", and not try to date out of their league.
FAANG companies offering industry leading compensation packages and prestige are in a position to be able to hold out for the best (even if their interviewing practices may fail to achieve that), but most companies are just looking for someone that checks the boxes and seems like a good fit.
Employers trying to hire for fit and culture has resulted in the most inhumane and counterproductive processes I've witnessed. If that's what you really want to do, let the person work for at least a month in the team.
Provisional hires (which often exist in theory but they're pretty much a formality in general) don't work for the most part. Lot of overhead for the company. And in many cases you're asking for the candidate to quit a job and possibly relocate on the possibility they'll get a new position assuming that they click in a short time interval.
Relocation used to be pretty common for professional jobs. Don't know about today when there's more remote work. And maybe companies aren't as willing to pay for in general.
There's even more overhead on the people being provisionally hired.
Yes, sometimes things just don't work out. But, if someone quits a job and maybe relocates, that's a big personal cost. It's just the way things work in some limited contexts (e.g. professional sports) but it's not and shouldn't be the norm.
I suppose you can give a huge sign-on bonus with no claw-back provision, but that's never going to happen in most cases.
I'm fully convinced the way to make better hires is to invest more, which will be more expensive. Which wouldn't be a problem unless we expected something else. It starts with quitting pretending the current process is working, or even close to optimal.
Has hiring ever really worked, anywhere? Especially as roles and need evolve? I guess you could argue that it sort of did, apropos of a play I saw last night on the astronaut program--and maybe the military in many cases more broadly.
But, in many cases, I'm not sure how I, as a candidate for a tech job, would feel about a company offering me $200K--no strings attached--with the proviso that I statistically only had a 25% chance of making it through the next 6 months. (And is that really long enough anyway?)
There are tournament-style professions. But I'm not convinced most professional jobs are or should be among them in general.
yes. Best place i worked at - we hired only by internal references and only people from our University. Up until the company grew around 200 people. We didn't do technical interviews, just a short talk. And we were among top employers, including salary-wise.
When you have high trust a lot of other processes become unnecessary. When that trust is broken, and surely a lot of grifters BSed their ways into jobs, that’s when all kinds of barriers were added.
I agree. But what I mean is: that's not how it's perceived in the current interview structure, which lasts maybe 45 minutes or so. Ultimately, going down a dead end means you'd now have 30 minutes to find the right solution and code it up. So the oracle (the interviewer) would probably help you realise sooner that it's a bad idea, so you don't waste your time. That's assuming they know the problem and solution well; if they don't, you'll just lose them and burn through your time.
In a 2 hour pair programming session on an 'unsolved' problem (like an open issue / minor bug / minor feature in a public repo), yes, it will likely not matter if you tried a bad idea, and would both be more realistic and a positive signal.
this is an interviewer problem, unless the candidate is totally silent. A candidate that can't ask questions isn't going to succeed, anyway.
If the candidate will communicate with me, I will offer them a LOT of guidance. It is still very, very easy to tell who knows what they are doing and who does not. You give them a basic but domain-specific task, you give them whatever extra context they need to do it, and you watch them hammer out the code.
It should be a task that is sufficiently familiar to the person applying to the role that they do not need to do -much- looking at docs, and as the interviewer you should be prepared to help them quickly find the docs you already know they will need -- you designed the task after all -- so that they don't waste time with that.
What's important is that they ask for docs when they need them, and that they can understand them, quickly, and use them. It will be obvious if they are using AI because of how long things will take. It will be obvious if they don't know when to reach for documentation, and it will be obvious if they cannot understand the documentation.
Then, they should write a test for their solution. This weeds out 95% of candidates. Talk to the other 5% and you'll find someone who can both actually write code and also discuss design.
A lot of good people will close-up and not voluntarily ask you any question if you put them under pressure. (What they automatically are on that exercise.)
What is not to say that you are making anything wrong. But watch for bias there.
> A lot of good people will close-up and not voluntarily ask you any question if you put them under pressure.
This sounds like the kind engineer who won't push back or ask for clarification on unclear requirements - and happily spend a month solving a problem the business doesn't have.
So maybe seeing that at interview time is a good thing?
Might not mean the candidate doesn't fit - but can clarify what kind of roles would work?
> So maybe seeing that at interview time is a good thing?
Perhaps, but an interview is a fundamentally different environment than day to day work.
There's no way to solve the interview/interviewee problem though, the whole thing is impossibly fucked and is going to have false positives/negatives no matter what.
I do 1hr pair programming interviews for my company and you have to strike a balance between letting candidates think through the problem even when you think it won't work (to see their thought process and maybe be surprised at their approach working/see how quickly they can self-correct) and keeping them on track so that the interview still provides a good signal for candidates who are less familiar with that specific task/stack.
I'm also not actually testing for pair programming ability directly, moreso ability to complete practical tasks / work in a specific area, collaborate, and communicate. If you choose a problem that is general/expandable enough that good candidates for the position are unlikely to go down bad rabbit holes (eg for a senior fullstack role, create a minimal frontend and api server that talk to each other) it works just fine. Actually with these kinds of problems it's kind of good if your candidates end up "studying" them like with leetcode, because it means they are just learning how to do the things that they'll do on the job.
> maybe even on an actual feature
I don't think this would work unless the feature were entirely self-contained. If your workaround is to give the candidate an OSS project they need to study beforehand, I think that would bias candidates' performance in ways that aren't aligned with who you want to hire (eg how desperate are they for the role and how much time outside of work are they willing to put into your interview).
> if you wag your tail chasing a bad end for 15 mins, this is a fail in an interview
Eh, if it's a reasonable bad end and you communicate through it, I wouldn't see it as a fail. Particularly if it's something I would have tried, too. (If it's something I wouldn't have thought of and it's a good idea, you're hired.)
I did a couple of rounds of this with my manager as the interviewer. Personally I really liked the process, and the feedback I got from the candidates was positive (but then again it always would be).
What worked well for me was that I made it very clear to my manager, a man who I trust, that I would not be able to provide him with a boolean pass/fail result. I couldn't provide him any objective measure of their ability or performance. What I could do was hang out with the canditates for an hour while we discussed some concepts I thought were important in my position. From that conversation I would be able to provide him a ranking along with a personal evaluation on whether I would personally like to work with the candidate.
I prepared some example problems that I worked for myself a bit. Then I went into the interviews with those problems and let them direct those same explorations of the problem. Some of them took me on detours I hadn't taken on my own. Some of them needed a little nudge at times. I never took specific notes, but just allowed my brain to get a natural impression of the person. I was there to get to know them, not administer an exam.
I feel like the whole experience worked super well. It felt casual, but also very telling. It was almost like a focused date. Afterwards I discussed my impression of the candidates with my manager to ensure the things I was weighing was somewhat commutable to what he desired.
All in all it was a very human process. It must have taken an enormous amount of trust from my manager to allow me the discretion to make a subjective judgment. I was extremely surprised at how clearly I was able to delineate the people, but also how that delineation shifted depending on which axis we evaluated. A simple pass/fail technical interview would have missed that image of a full person.
I've (unfortunately) been interviewing the last two months and the main pattern that I've noticed is that a) big companies have terrible interview processes while b) small companies and startups are great at interviewing.
Big companies need to hire tons of people and interview even more so they need some sort of scalable process for it. An early stage startup can just ask you about your past projects and pair program with you for an hour.
I hear this all the time, but I have yet to experience it. It may be because the small companies that I interview with are all startups, but I have yet to be able to get a call back from any other kind of small company. And the startups I do interview with have a full FAANG interview loops.
There seems to be a weird selection bias that if you're FAANG or FAANG adjacent these small companies aren't interested.
At a former gig we had a newly hired ex-facebook employee give notice within a month because she didn't like that dev setup had bugs that devs themselves had to fix. At fb they obviously can spend millions of dollars for a whole team that ensures that working dev env is always a button click away, a startup (even a scaleup) usually can't afford to. This is just one example out of many I can tell...
And I’ve heard just as many horror stories about companies hiring from small companies that the engineers haven’t kept up with engineering, culture and practices and are coding like it’s 2004.
Also, those types of stories tend to pop up with any engineer who’s only worked at a single place.
My point isn’t that there’s not bad engineers at Facebook it’s that there’s bad engineers everywhere and filtering based on random signals like this is not useful.
The sad truth of hiring is that you can't afford to interview everyone, and have to keep in mind that prospective employees are showing you a different version of themselves than they'll actually bring to work.
Especially in a small company where your hiring manager may also be busy with development and sales, and not have an HR department to run the process for them, you're much better off interviewing candidates you are 50% sure of being a good fit vs 5%. Personally I prefer interviewing candidates coming from FAANG-ish companies and often make exceptions for candidates that demonstrate exceptional skill/interest, but when you can only interview 1-10% of your applicants you have to prioritize those who are likely to succeed at your company (keeping in mind implicit bias and such).
> filtering based on random signals like this is not useful.
In aggregate it most likely is useful for those companies.
> And I’ve heard just as many horror stories about companies hiring from small companies that the engineers haven’t kept up with engineering, culture and practices and are coding like it’s 2004.
Big cos can afford to onboard their engineers for months, sometimes years. Startups usually can not
>”There seems to be a weird selection bias that if you're FAANG or FAANG adjacent these small companies aren't interested.”
Many smaller companies have noticed that former and wannabe FAANGers are looking for FAANG-type jobs, and are not good fits in their niche. Small companies often have more uncertainty, fewer clear objectives, less structure, and often lower pay. They’re not a good substitute for megacorps.
The problem is that many smaller companies have hired people from FAANG, and had them quite after a short tenure, so they're unwilling to try their luck again, as it's just not worth it. You may be different, but they've heard that story before, and had it not work out.
As someone that never had a desire nor ever made an attempt to work at any of those companies, do you mind elaborating on the mentality of such places?
I'm just your boring below-average to average dev, so I know I'm not cut for those types of places, but it never truly bothered me anyway. Any reason that I can personally think as to why I would work for such a company would either be due to my own egotistical desires or for monetary reasons, but those were never strong enough to actually compel me.
I am just mainly curious about two things:
1. Is working at those places all it's cracked up to be?
2. Assuming one had to work hard to get into such companies, was the juice worth the squeeze?
I've often wondered if one's experiences for these companies is often something akin to the old advice of, "Don't meet your heroes." In other words, was the conflicting dyad of expectations vs. reality present?
It has turned into something similar to what people in trading companies on Wall Street deal with. Constant grind, unrealistic expectations, and projects done in order to get a promotion instead of because it provides value to the customers or the business.
That said the amount that you make is insane some of the smartest engineers I’ve ever worked with have been at these companies and a lot of them have really strong engineering cultures, and standards.
The current work environment seems designed to use up bright young engineers, and burn them out within a few years. This is a significant shift from 15 years ago, where it was a much more sustainable place to be.
I could see that happening, especially given that they're young and feel powerful, nearly invincible. I experienced a similar attitude form younger folks outside of FANG, some kind of derision of experience and advice from people who have been around and choosing instead to obsolete it. After the first burnout may of them become a lot more thoughtful and humble. I may have been like that too when I was young, I don't remember being exactly like that but maybe I was just not aware. I still did respect experience and I still do to this day.
Sure, but in this context, your FAANG experience is a negative signal for people who don't know you well yet. It's unfortunate for you, but a genuine factor you now need to account for.
Your path through will probably look like having the luck of breaking in at one of these kinds of companies, and then staying for several years to demonstrate earnest commitment/fit while building a new network of connections, and then leveraging those connections to get more opportunities if it becomes necessary to do so. If you have connection from your previous non-FAANG work, that's probably your best route.
It won't happen overnight and you'll always be at a disadvantage when you find yourself applying through resume portals. Good luck!
In my time at tier one companies I have worked with the best engineers I have come across in my entire career (even the worst engineers were more than competent) who were working on deep issues that could affect the revenue of the entire company because they’re laser focused on providing value to the business, instead of doing engineering for engineering’s sake. I have grown by far more in these kinds of roles than I have anywhere else because the kind of problems you encounter at such a high scale just don’t exist elsewhere. And most of them have been there for at least five years if not longer you don’t make those kind of contributions to accompany without a long tenure.
> In my time at tier one companies I have worked with the best engineers I have come across in my entire career
You’re throwing a giant red flag right here. First of all, FAANG isn’t “tier one” except to people who idolize these companies. More agile startups are trying to disrupt these dinosaurs and do not thing very highly of them. Many of us who have worked with FAANG and ex-FAANG engineers were not impressed.
I mean, I'm just sharing the practical ground truth of how a resume like yours effects recruiting in certain contexts.
Just like there are innumerable brilliant, effective engineers who would contribute tremendously to a FAANG but don't suit the modern interview funnel (leetcode, etc), smaller companies surely do miss out on strong, suitable FAANG engineers in anticipation of negative experiences they've had with others.
There are a lot of people who accumulate FAANG entries on their resume and many of them really don't suit smaller companies for a number of reasons.
Honestly, though while I'm only seeing a very narrow picture of you here, it sure sounds like you see these "tier one" companies as a desirable place to work, with prestigious colleagues and profound learning opportunities on high scale problems that just don't exist elsewhere, and surely for much more money. Are you sure you're really going to be happy somewhere else? Or might you get restless? That's precisely the kind of concern these smaller companies carry when seeing FAANG stuff on a resume, and it doesn't seem like it should be baffling that they would do.
I definitely used to. The work culture and attitudes, particularly of management, passed the breaking point for me a few years ago. I realized work was not my whole life nor did I aspire to that.
>"And most refused to look at anybody deviating from their ideal background in my experience."
This is often because the culture of job-hopping for better pay every 18 months has eroded the willingness to pay for training or adaptation. Why pay for someone to learn if they're just gonna leave soon; the pre-trained person is a better deal if you'll have to pay to retain anyway.
Which was caused by cost cutting measures, MBA disease, in companies to begin with.
We’re just seeing the end of the cat and mouse struggle that’s been going on since the 60s. And massively accelerated in the 80s.
It’s unfortunate for companies though because they’re the ones that will lose out in the end when all the experienced people start retiring and they have no one to hire.
It’s an untenable position to not train people, period. There is no schooling you could go through that would educated junior dev to the level of a senior dev. And it’s the same for any other role. Experience is not optional.
I think the primary stimulus which creating the “job hopping culture” was actually the hot labor market for software developers. Other fields experienced real ‘cost-cutting’, without resulting in a lot of ‘job-hopping’.
I agree that this situation is undesirable, but it seems to be stable, somewhat like the result of repeated play of the prisoner’s dilemma.
That definitely massively accelerated it but you’re looking way too short term that’s only been in the last 10 to 15 years.
I agree that other industries are not YET at the point where software is , but you’re not looking hard enough if you don’t see the short tenures compared to the 25-30 years they used to have.
And yeah, it might be in an equilibrium now, but how long can it stay in an equilibrium? I’d guess at max 10 to 15 years.
I'm guessing the majority of people now in their 50s and 60s in computer-related careers had very eclectic jobs before settling down in computer-related stuff. After all, many never used computers at all until college or beyond.
My understanding is even in the early 2000s it was pretty much just firmware versus desktop software with a small niche for Mac developers.
Edit: my point was not that specialized software applications didn’t exist. It was that people were expected to be able to jump from stack to stack when they change roles in a way that has disappeared from modern job applications.
Well, and mainframes. And trading and financial systems. And numerical/scientific computing. And network services. And web sites and e-commerce. And flash, java applets, and browser plugins. And control systems. And operating systems and tooling. And cell phone applications. And games. And video/image/audio/music processing. etc etc
It was probably about as hard to move between those domains now as it is today. Which is to say that it's pretty hard and needs some concerted, non-trivial effort in shaping your experience and how you present it before trying to make a transition, and often either some kind of inside reference to vouch for you or an employer that was especially hard up for candidates. Or else an employer that straddle multiple domains and actively supported internal transitions.
Depending on what you could bring attention to in your prior experience and the size/needs of the new orgs you seeking to move to, certain transitions were more feasible than others, but you could easily spend decades working in mind-numbing enterprise applications while wishing for opportunities in game development or trading or whatever and never get your resume so much as looked at. (And vice versa, even, for those who dreamed to "retire" into the supposed quiet of enterprise apps or government IT or whatever)
I basically agree with your edit. There was a lot more fluidity among roles and even just moving into computer roles from other engineering (and even non-engineering) fields. But that's not really what you wrote initially.
I've had a few good experiences with interviews at small companies and startups, so they do exist.
But I have also had really terrible experiences, similar to what you've mentioned. Sounds like you've just gotten unlucky and gotten the terrible ones.
I'm not talking about being watched closely, I'm talking about regular work and then making a decision. You have to leave people space to do their thing if you want to see what they're capable of.
Sure, that would be even better. But how would that even look?
In the best case applicants needs to apply multiple companies. Companies need to interview multiple applicants and have a way to compare those applicants.
Those are the most basic constraints I can think of. How do you make that cost tens of hours for each round?
For me as a job applicant even in the best case I would need to do 3 to 5 interview interviews. The same is true for companies in the best case it will take at least 3 to 5 interviews to find somebody. Are they supposed to have 3 to 5 temporary staff for weeks at a time?
How much time should that take per interview? How would somebody that currently has a job manage that kind of time commitment?
You seem stuck on the idea that you need to verify an employee to hell and back before investing anything, that's not what I mean by committing. If they look like they might be a reasonable fit, give them a month to prove it, then you'll know for sure. One interview per person should be enough to make that decision.
In a small company, you can tell your buddy “just have a chat with the candidate and if you like them and you think they can do the job, hire them”.
If the person interviewing your candidates messes up, you’ll know soon enough. In a large company, the bad people will take over and your company is dying a slow death.
That approach doesn’t work on a large scale. Some interviewers are too nitpicky, elitist, others approve anyone who uses the same language as them for side projects. Some are racists, sexist, or have other kinds of biases. Some might have a crush on the candidate. Sometimes the interviewer thinks about their own task while they squeeze in an interview. In some countries, “undoing” a bad hire is hard, so they need to make sure that the candidate can work on any team (or at least on multiple teams reasonably well).
IMO for large companies it makes sense to standardize the interview process.
Also, in my opinion grinding leetcode is also a good personality check for FAANG hires: it shows the candidate can suck it up, study hundreds of hours, and do whatever they need to do to pass an arbitrary test, even if they themselves think it’s a broken process. The larger the company, the more this quality matters in candidates as they will need to deal with a lot of things they will probably not like.
> In a large company, the bad people will take over and your company is dying a slow death.
Why is this the assumption. I would rather say any big org. is converging towards the average talentwise by necessity. It is like Hawaii can't have 100 olympic level swimmers no matter the recruiting proces.
I'm quite conflicted on this. While I do not think one needs to remember/memorize a bunch of brainteasers or past computer scientists'/mathematician's PhD discoveries in order to build CRUD applications.
However, I do feel like there is perhaps some amount of truth to the thought behind the interview questions, no? As in, I would imagine someone that could invert a binary tree in 15 minutes on a whiteboard could probably learn React. However, I am not sure everyone that can learn React can invert a binary tree in 15 minutes on a whiteboard.
However, maybe I am projecting my own insecurities because I wish I could invert a binary tree in 15 minutes on a whiteboard as well as being able to solve all those other problems.
No, it's not actually about mastery of binary trees or other abstract computer science concepts.
leetcode is really a culture fit test and success points to the combination of "smarts", diligence, and conformity to some acceptable degree. It shows you have some baseline of familiarity with computing, can focus on an arbitrary task to pursue a goal, and will conform to an arbitrary process when it's asked of you.
Those are genuinely essential skills in an organization with 1000's of white collar professional workers.
More precise insight is gained when the test covers a binary tree inversion than that were it just some contrived logic puzzle like the LSAT or a critical reading exercise. The computer science bit does provide some signal and isn't completely arbitrary, but it's only a small part of what's being evaluated.
Among otherwise strong engineers, it tends to filter out the especially willful, prideful, independent, meandering, creative, and pragmatic ones. These personality types can be extremely valuable in some work environments and can still sometimes in through leetcode challenges, but spoil big bureuacratic systems like FAANG's when they're overrepresented.
The culture thing is huge. If you can’t find a few good leet code problems to enjoy, I don’t know if we are interested in the same field.
Having done a lot of interviews at least 70% of the time it’s filtering candidates who just aren’t very technical and haven’t studied computer science. The kind of people who hate reversing a linked list are the kind of people who haven’t touched a polynomial since high school.
It’s not unreasonable for a junior tech interview to expect you studied something like CS or EE. It’s a blessing that our field is open to all to give it a shot, but if that doesn’t describe you - you need to recognize that there is effort and study to fill that background.
But I'm talking about the idea of pair programming for an hour or two. Why can't the standardized test still involve that? What doesn't "scale" about it?
I think it’s easier to compare candidates who might have been interviewed by different people possibly weeks apart if you have a coding challenge… could they solve the problem we gave them, if yes how quickly and how well. Interviewers can write their notes, the session can be recorded and the committee can compare candidates relatively fairly.
Comparing candidates based on how they “vibe” with the interviewer during a pair programming session is a recipe for lawsuits and bad hires.
I’m just speculating here, I don’t have any significant hiring and interviewing experience
If you're worried about interviews weeks apart and cross-team testing, then the most that can hurt efficiency is making you do the test with the team they'll be imminently hired into, which means you're on par with the small company, nothing lost nothing gained.
Avoiding pair programming for the reason you listed sounds like lawyers getting in the way, not scaling. But yeah we'd need to hear someone say if that's actually happening.
> If a startup can spend 20 man-hours filling a single position, why can't a big company spend 1000 man-hours filling 50 positions?
Because big companies are run by bean counters and they also don't require the same kind of talent that is useful to startups. There's less competition for hyper-specialized seniors and middle of the pack generalists.
> small companies and startups are great at interviewing
Small companies have the benefit of the pressure to fill a role to get work done, the lack of bureaucratic baggage to "protect" the company from bad hires, and generally don't have enough staff to suffer empire-building.
Somewhere along the line the question changes from "can this candidate do the job that other people in this office are already doing?" to "can this candidate do the job of this imaginary archetype I've invented with seemingly impossible qualities".
Do we have different definitions of "marginally qualified"? Idk, I feel I'm a decent engineer - I can certainly do whatever leetcode medium they throw at me, as much as this counts of anything - and can actually code, but I still get maybe 1 callback per 50 applications.
Does "marginally qualified" mean "Ivy League Competitive Programmer PhD" or something?
> Do we have different definitions of "marginally qualified"?
Not every candidate is an interview. I recently hired. I got 90 applications and from these, 80% were an instant "No". They didn't match the job description or had no permit to work in my country. I invited the rest. Simple interview, pair-program a dead simple App with a prebuilt skeleton with me with any framework of choice. Make one GET request, render it and realize one needed optimization which needs to be implemented. 90% (I'm not joking) of candidates failed the first task. In half an hour, they couldn't send a GET request and translate that into JSON. All were allowed to google, open any documentation they liked. Of all 18 who failed, 16 asked me if they could use an LLM for this task, which I denied.
The GP is still passing through hundreds of people, dozens of them capable, until he reaches somebody that convinces him of their competence. You were passed down because you weren't convincing enough.
Or maybe he is getting resumes from a channel that has been victim of machine-gun filling, and there indeed thousands of incompetent people posting resumes into every channel and just half a dozen real applicants.
TBF, I have no idea how to fix either one of those problems. Hiring is just completely broken.
I have only a resume to convince them. I have job experience at major companies, with examples of what I've built when there, personal projects I've made, a github link, a great GPA from a good school.
And I know similar Junior-mid people in the same boat. We can all do Fizzbuzz, we've all built things, and somehow we're not getting interviews, but people that can't do Fizzbuzz are.
Do thousands of incompetents also machine-gun apply to, say, mechanical engineering, accounting, marketing, HR, or finance gigs? Is it just tech?
> Do thousands of incompetents also machine-gun apply
Enough that it's HR or some automated software that first screens your application.
> Something isn't adding up.
Yes. The pipeline "posting > application > screening" is now completely broken. In your case it's quite possible it's HR and screening software that your resume is not convincing. I have been hoping for anecdotes and studies in this direction (from people who have access to the HR and/or software screening and are inclined to report) - but it's at least not common. What we do hear from, is tons of people who can program and who are not getting even first interviews.
Big companies and small need to agree on some standards and create qualifying exams that they will actually accept as proof of competence. Degrees somehow don't prove anything, experience doesn't, blah blah blah. It's exhausting to have to prove to interviewers that I'm not mentally disabled at every turn and it's a waste of time for everyone.
Create certifications that actually count for something and aren't just a blip on a resume that may tick a box, but will actually move you past technical trivia questions. I know some people have a deep repulsion to this and I think it would be fine to have a technical interview gauntlet for those that choose not to engage with any type of certification and a simplified interview format for those that have passed the prerequisite tests.
I don't care how long, rigorous, or ridiculous the tests are. Just agree on some effing standard.
Watch out for what you ask for. Plenty of big vendors have certification programs. ... And for some combinations of field and vendor, they are red flags - rather than pre-validation. That is, far too many applicants have the certification but do not have the grounding knowledge without which the certification is sort of useless - potentially more dangerous than validating.
>The best interview process I've ever been a part of involved pair programming with the person for a couple hours... You never failed to know within a few minutes whether the person could do the job
There is something funny about the "best interview process" taking "a couple hours" despite giving you the answer "within a few minutes". Seems like even the best process is a little broken.
Lightly ironic indeed! Though I’m not sure “broken” is exactly the word I’d choose.
I can only speak for myself, but I imagine myself as a candidate approaching a “couple of hours” project or relationship differently than I would a “few minutes” speed round. For that matter I can think of people I know professionally who I only know through their behavior “on stage” in structured and stylized meetings of a half hour or an hour—and I don’t feel like I have any sense at all of how they would be as day-to-day coworkers.
If we sat down to work together, you’d probably have a sense in the first few minutes of whether or not we were going to work out—but that would be contingent on us going into it with the attitude that we were working together as colleagues toward a goal.
That's mainly because there were multiple pairing sessions, and even if you knew the person was going to pass, there are still a couple more people who need to meet them, and a schedule to make sure they're available to do that. Plus due diligence, etc.
Nor am I saying it was a perfect system, just the best I've seen in terms of results.
The biggest victims of these non-scalable process is people without a good network. As an intl PhD student, I am that person.
So now I have this weird dynamic: I get interview calls only from FAANG companies, the ones with the manpower to do your so called "cursed" scalable interviews. But the smaller companies or startups, ones who are a far better fit for my specialized kills, never call me. You need to either "know someone" or be from a big school, or there is zero chance.
I've been interviewing recently and got through to the last round (of five...) with an interesting company. I knew the interview involved pairing, but I didn't expect: two people sitting behind me while I sat on a wobbly stool at a standing desk, trying to use a sticky wired mouse and a non-UK keyboard, and a very bright monitor (I normally use dark mode). They had a screen recorder running, and a radio was on in the next room.
I totally bombed it. I suspect just the two people behind me would have been enough though.
Pairing on something close to whatever real work they'd be doing, but familiar to the applicant is my favorite way to evaluate someone (e.g. choose a side project, pre agree adding a feature).
I don't care if someone uses modern tools (like AI assists), google, etc - e.g. "open book" - as that's the how they want to work. Evaluating their style of thinking / solving problems, comms, and output is the win.
Very few people doing this sort of interview (they tend to be our best, most empathetic developers) are likely to cut a multi-hour planned process short after a few minutes. It will eat at least an hour of their (very expensive & valuable) time.
Also how am I supposed to filter the 100's of AI-completed assessments? Who gets this opportunity?
We didn't do assessments (if by that you mean take home assignments). This was partly a solution to that, since nobody thought they were a good idea. If you mean the phone screen, I think that would be a problem, yep, but it wasn't an issue back in 2016. Having them pair would weed out cheaters, but we would have to figure out a way to weed them out during the screening, I agree.
We also did not require the employers doing the interview to be our most senior team members. They probably did it more often than most people, but often because they volunteered to do it. Anyone on the team would be part of the loop, which helped with scheduling. And, remember, we were working on actual tickets, so in a lot of cases it actually helped having the candidate there as a pairing partner.
For a little extra detail, the way we actually did it was to have 2-3 pairing sessions of up to 2 hours apiece. At the end of the day, all the team members who paired with the candidate had to give them the thumbs up.
This. Interviewing for a sr dev position with a web app, backend stack is the bog standard java, spring, SQL abstracted away via JPA. We did a first screen, then the tech interview was two of their senior devs shoulder surfing me as I built a simple API. We chatted, I built, they asked questions, I defended my decisions (sometimes successfully, sometimes gracefully conceding defeat), they left knowing that I was who my resume said I was and the reminder that popped up in the middle of the interview to feed my sourdough starter showed them that I'm a culture fit.
I think you're onto something with that last paragraph but I want to try being a bit more generous with why things are the way they are. The question seems to be "When there are hundreds of applicants how do we give everyone a fair shake without hiring an entire team of devs who do nothing but interview?" From that perspective the intentionality is different and even sensible but the end product is likely to be the same. Even when someone is chasing a metric it's because someone else wants what's best and has decided that metric is a sensible way to make that happen. At the end of the day they really do want to hire the best candidate out of a pool whose size is extremely variable and that's challenging.
I work at a company which has 11 engineers and competes with companies with 100s. The hiring process was a screening call with the CTO to not waste the prospective team's time, then a call with 2 of my prospective colleagues to gauge competence and cultural fit. Since then I have been involved in hiring most of the team I work with now. The CTO is one of the most competent engineers I have ever met and he designed this process. He also has very high EQ. One of the points I sell to prospective hires is him as a person to work with, as well as our team. He has also flatly denied people I suggested before and that's fine.
I have been here 5 years now and I'm working with the most competent team I have ever worked with. My take away from this is that hiring doesn't need to be commoditised and scale, it just needs to find good people and give them an opportunity to show you that you do or don't want to work with them.
I've been a proponent of pair programming since the early days of Agile, when it was still seen as part of extreme programming. Unfortunately, it’s not often employed in workplace settings.
With that said, would your perception of the interview remain positive if the outcome had been negative?
A common challenge across all interviews is a mismatch in personal dynamics, which can significantly impact the experience for both participants.
Consider a scenario where a senior developer, who prefers simplicity, is paired with a mid-level developer who is still captivated by complexity.
Or a "just start typing" person is with a "mull it over first" person. By the time I am typing code, I want to have 90% of it already completely worked out (at least till I type a "c.Lock()" and suddenly realize I hadn't considered thus and so synchronization issue.
Similarly, in general the best interviews I've ever been part of (either giving or being) turn into discussions where people's experience, opinions, and stories get aired (going both ways). You eventually get a good sense of each other and things get more relaxed when you both realize that you know what you're talking about (this is harder for Jr roles, though).
Being peppered with questions very rarely gives any insight.
For junior roles, you want to interview for intelligence and shall we say an interest in learning rather than specific skills.
Even for senior roles, that's what I want to interview for, although it is true at times a business case can be made for someone that is good at some specific complex skill and doesn't need to listen to other people to do ok work.
I used to love getting to know the interviewer and doing things like that but IMO the market has shifted fundamentally on both ends for this to be effective anymore for most SaaS roles. This is anecdotal for US/Canada tech market over the past 10 years so YMMV.
Developers Side:
Since developers don't have job security anymore (at least for those who work on common languages like Go, Python, Java and Typescript) they are better off learning and keeping in touch with leetcode and system design questions, looking for new opportunities and interviewing in "batch mode" when looking for a job. The idea is to clear as many interviews as possible using the same concepts, get in and make money asap before you get laid off. No incentive for collaboration or for fulfilling but esoteric stuff like Haskell and Scala. Career security > Job security.
Companies Side:
On the other end software companies have less trust in developers staying long term so they want to make the interview process as quick and risk free as possible. In essence they are betting that by perusing 100s of resumes and hiring someone who seemingly knows CS concepts they can get some value out of them before they leave. Standardized tests/vetting > team fit.
TLDR; The art is gone from this job, its become akin to management consulting or investment banking. Quality and UX seems to be regressing across the board as a result.
I meant the “grind”, short term profit mentality of SWE market has become similar to professionals in those fields, not that any of these fields are similar.
I think what makes it work is that our code pair is pretty low stakes. I was told that I didn’t have to finish the problem and I was free to use whatever tools or language I needed. They just wanted to see how I work and collaborate.
Thats what we did, pair program on some real production code and tickets. This way the person could get a feel about what they potentially were walking into and you get a good idea of how they think and approach problems.
Teams are really sleeping on code reviews as an assessment tool. As in having the candidate review code.
A junior, mid, senior, staff are going to see very different things in the same codebase.
Not only that, as AI generated code becomes more common, teams might want to actively select for devs that can efficiently review code for quality and correctness.
I went through one interview with a YC company that had a first round code review. I enjoyed it so much that I ended up making a small open source app for teams that want to use code reviews: https://coderev.app (repo: https://github.com/CharlieDigital/coderev)
This is harder than it sounds, although I agree in a vacuum the idea is a good one.
So much value of the code review comes from having actual knowledge of the larger context. Mundane stuff like formatting quirks and obvious bad practices should be getting hoovered up by the linters anyways. But what someone new may *not* know is that this cruft is actually important for some arcane reason. Or that it's important that this specific line be super performant and that's why stylistically it's odd.
The real failure mode I worry about here is how much of this stuff becomes second nature to people on a team. They see it as "obvious" and forgot that it's actually nuance of their specific circumstances. So then a candidate comes in and misses something "obvious", well, here's the door.
You can do code review exercises without larger context.
An example from the interview: the code included a python web API and SQL schema. Some obvious points I noticed were no input validation, string concatenating for the database access (SQL injection), no input scrubbing (XSS), based on the call pattern there were some missing indices, a few bad data type choices (e.g. integer for user ID), a possible infinite loop in one case.
You might be thinking about it in the wrong way; what you want to see is that someone can spot these types of logic errors that either a human or AI copilot might produce regardless of the larger context.
The juniors will find formatting and obvious bad practices; the senior and staff will find the real gems. This format works really well for stratification.
> no input validation, string concatenating for the database access (SQL injection), no input scrubbing (XSS), based on the call pattern there were some missing indices, a few bad data type choices (e.g. integer for user ID), a possible infinite loop in one case
I'd say all this stuff is junior-level (maybe ~mid for things like user ID integers). It's just a checklist of "obvious bad practices", it doesn't require experience.
The senior stuff is much higher-level: domain modelling, code architecture, consistency guarantees, system resilience... system design in general.
You can do all of that in a code review; the point is that it actually allows for better stratification because you can incorporate different challenges in a reasonable time frame and without having to do take homes and get working environments (you'll end up reviewing their code anyways in a followup session).
You can do it in a real code review. I think his point was that you can't do stuff like "instead of loading a YAML file at runtime this should be generated during build time using the existing infrastructure we have here" type stuff.
But I'm not sure you really need to in a job interview. It's not like you can do that with any other interview method anyway - leetcode also doesn't really touch high level architecture type stuff, and take home problems are also too small (or they should be anyway!)
In my experience you only learn how good developers' architectural taste is by working with them for a long time.
In a previous job we did code review interviews. And went the route you said due to the problem I said. And yes, it's a lot better. But what also happened over time was that the bar slowly raised. Because over time the "harder" pieces of that session started to seem rote to interviewers, they became viewed as table stakes.
Mind you this is true of any interview scheme that has a problem solving component to it. I'm not saying that the code review style is extra bad here, just that it doesn't solve this problem.
I think the way to avoid the interviewer's expectations being raised over time is to write down some guidelines for what a successful candidate should be able to do. Even if you don't know how high to set the bar at the beginning, once you've hired someone you'll have at least one example of a good answer.
In theory, you can do code reviews without larger context if the code is carefully selected. Apparently, some companies think any badly written code from their code base can just be selected though.
It's not so hard. One of the interview stages I did somewhere well known used this.
Here's the neural net model your colleague sent you. They say it's meant to do ABC, but they found limitation XYZ. What is going on? What changes would you suggest and why?
Was actually a decent combined knowledge + code question.
There are so many interesting ways to use code reviews like subtly introducing defects and bugs and see if people can follow the logic, read the code, find where the reasoning comes up short.
I like the code review approach and tried it a few times when I was needed to do interviews.
The great thing about code reviews is that there are LOTS of ways people can improve code. You can start with the basics like can you make this code run at all (i.e. compile) and can you make it create the right output. And there's also more advanced improvements like how to make the code more performant, more maintainable, and less error-prone.
Also, the candidates can talk about their reasoning about why or why not they'd change the code they're reviewing.
For example, you'd probably view the candidates differently based on their responses to seeing a code sample with a global variable.
Poor: "Everything looks fine here"
Good: "Eliminate that global variable. We can do that by refactoring this function to..."
Better: "I see that there's a global variable here. Some say they're an anti-pattern, and that is true in most but not all cases. This one here may be ok if ..., but if not you'll need to..."
100% it is more conducive to a conversational exchange that actually gives you better insight into how a developer thinks much more so than leetcode.
Coding for me is an intensely focused activity and I work from home to boot so most of the time, I'm coding in complete silence. It's very awkward to be talking about my thought process while I'm coding, but not talking is just as awkward!
Some of the most interesting interviews that I felt like accurately assessed my skills (even non-live ones) where debugging and code review assessments. I didn't get offers from these cos later on because I failed the leetcodes they did later in the process but I felt the review interviews were a good way to be assessed.
I loved the idea of code reviews interviews, i've had several good ones, until yesterday when I had my first bad code review interview.
They asked me to review a function for a residential housing payment workflow, which I'm unfamiliar with. From an actual snippet of their bad production code (which has since been rewritten). In Go which I've never used (I've never professionally used the language that doesn't have error handling built-in, for example).
I had to spend more than half of my time asking questions to try and get enough context about Go error handling techniques, the abstractions they were using which we only had the import statements to and the way that the external system was structured to handle these requests to review the hundred lines of code they shared.
I was able to identify a bunch of things incidentally, like making all of the DB changes as part of a transaction so that we don't get inconsistent state or breaking up the function into sub functions, because the names were extremely long, but this was so far outside my area of expertise and comfort zone that I felt like shooting in the dark.
So just like any other interview style, they can be done very poorly.
Honestly, it was also a red flag for me that they don’t actually know what they want and have bad communication between leadership and engineering. Prior to this interview I was already on the fence about them.
They don’t work mostly in Go. Even the interviewer said that he’s vaguely familiar with this area of the code, but he doesn’t work and Go. They work mostly in Kotlin and they explicitly are advertising for solid generalists.
I don't know. A cold code review on a codebase they never saw is not super informative about how the candidate would interact with you and the code once they're in known territory.
Yeah it's really tempting when you discover an interesting fact to think "that would make an interesting interview question" and turn the interview into some kind of pub quiz. Happens with all forms of technical interview though. I mean 90% of leetcode questions are "this one weird trick".
Yep, I've done a lot of SQL interviews and it is always interesting to see the folks who've crash and burned at code review and killed it at writing individual queries and sometimes the unexpected, the opposite happened, the person would fly through a code review and do really subpar on writing it, a signal I usually took to mean that the person was nervous as hell in the interview.
The two folks who showed this behavior I hired anyway (they were contractors so nbd) and they were excellent hires, so I really love the code review approach for climbing up bloom's taxonomy.
Company A wants to hire an engineer, an AI could solve all their tech interview questions, so why not hire that AI instead?
There's very likely a real answer to that question, and that answer should shape the way that engineer should be assessed and hired.
For example, it could be that the company wants the engineer to do some kind of assessment whether a feature should be implemented at all, and if yes, in what way. Then you could, in an interview, give a bit of context and then ask the candidate to think out loud about an example feature request.
It seems to me the heart of the problem is that companies aren't very clear about what value the engineers add, and so they have trouble deciding whether a candidate could provide that value.
The even bigger challenge is that hiring experts in any domain requires domain knowledge, but hiring has been shifted to HR. They aren't experts in anything, and for some years they made do with formulaic approaches, but that doesn't cut it anymore. So now if your group wants to get it done, and done well, you have to get involved yourself, and it's a lot of work on top of your regular tasks. Maybe more work because HR is deeply involved.
Well, unless you know sufficiently senior people. But I suspect that is a deeply unsatisfactory answer to many people in this forum.
My long term last, only technically-adjacent, job came through a combination of knowing execs, having gone to the same school as my ultimate manager, and knowing various other people involved. (And having a portfolio of public work.)
I suspect many people who don't have strong networks for whatever reason resent that. To which you could probably tack on not having gone to the "right" schools or having a public portfolio.
And just because you’re an introvert doesn’t mean you are incapable of building soft skills. Talking to people is absolutely exhausting for me, but I force myself to do it and practice at it because I know it is important for my career.
No question. Yet, social connection seems worth 10x or 100x competence in any particular circumstance and the effects compound. There are some real benefits from and needs to be prosocial and socially competent but I've regularly seen social competent but technically incompetent people advance far over technically competent but less socially agenda driving people (that are nonetheless socially competent). This only gets worse at scale and as you progress.
I love coding and do it reliably well with joy but as my career has progressed I've struggled more and more with getting a company to let me work at a "low level" or to navigating what seem like sociopathic behaviors to really contribute at my capacity.
Sorry for my lack of clarity, yes. Pure code and technical contribution, up to mentoring, as opposed to holding architecture summits, politicking, and the like. I've been pushed into management and socializing without regard to my willingness.
Well, they also promote people based on impact and, with rare exceptions, if you're holed up in a corner someplace you're probably not having a huge amount of impact.
Reality is that the ones quietly holed up in the corner are usually doing all the unsexy maintenance-type work that the extroverts don't want to do (because it's not sexy).
Nobody cares about that work... until it doesn't get done. And so, nobody doing it gets promoted.
Is closing deals on the golf course even still a thing these days? I suppose it probably is in some circles but I haven't seen it in a couple of decades of tech industry life when it was more likely to be fun runs or skiing.
But my broader point was that golf course socializing seems like mostly a different world today, at least in my tech circles, relative to other venues.
As one, I have to say there's really nothing about being an introvert that prevents one from being affable and available. The idea is that human interaction does not boost the introvert's energy the way it does the extrovert's, not that it's impossible.
Dealing with people and communication can be learned.
I get it. By nature I was very much an introvert except for certain scenarios when I was in my comfort zone until at least my mid 30s. I was an only child, the stereotypical short, fat kid with a computer growing up in the 80s (still short, became a gym rat, part time fitness instructor and only stopped the latter as my other obligations became greater). Horrible dating life and a bad first marriage before turning 35 (happily remarried since then).
It became apparent that to get ahead in my career, “codez real gud” was going to limit my career. I slowly learned how to “act like I like people”.
But you can only add so much value to an organization typing on a keyboard. There is a reason that every single tech company promotes based on “impact”, “scope”, “dealing with ambiguity”. Those all require soft skills.
Well put. I'm an introvert, I can't do math, I won't travel, etc. are all things that some people claim as if it's the unchangeable nature of things. If that's their chosen path, so be it. But they should understand it will probably be pretty limiting because the world they live in.
Yes they are unchangeable. I've tried many many times to break out of it; it works for a while but i revert back to my base behavior.
We know what kinds of temperament a dog has within few months of it being a puppy ( and who the puppy's parents are). Why would it be different for humans.
Claim that Our temperaments ( and our likes/dislikes for travel) are all learnt is a bizzare blank slate claim that doesn't track with my life experience and what i've seen in the world.
Because animals typically live in a way more static and uneventful environments, and they have much more limited mental capabilities?
Humans (and other animals) aren't a completely black slate - but unlike most animals, humans have very complex societies that affect their behaviors throughout their entire lives. A few years in a different environment start to change people. Kids (with their still-growing brains) adapt faster, adults - not so much, but the traces will be evident. Move a not-too-fucked-up Russian to the Pacific Northwest, and they will eventually start to smile now and then.
Also, thanks to the language, humans can think things up even when alone, drive themselves crazy in all the weird ways, then overcome all that self-inflicted stress and possibly develop some behaviors as a result.
Of course it's relevant, by definition of "temperament". The question is how much of our (very complex) behavior is biologically based and independent of learning.
For a cat, it probably plays a significant role. Cat behaviors are complex but still much more simpler than humans. And changes are rare. Although I've heard of a "lazy" apathetic cat moving into a house with giant outdoor catio and becoming drastically more active, almost like a different kitty.
I'm not sure about humans - how much of our behavior is a true temperament and how much isn't despite tending to not change throughout one's life. I've seen introverts becoming eager activists after they went through some bad things, like war and prison. I've seen people who were jumpy and always nervous becoming relaxed after many years in safety.
I hated small talk traditionally. It was very much a learned trait. One of the best conversation openers is “what keeps you busy?” and then ask open ended questions.
Ask about their favorite travel destinations or even what are some interesting things about where they live.
On the other hand, step outside your comfort zone and try something different so you have something to talk about interesting.
People can certainly decide that certain activities aren't their thing.
I don't want to push to give presentations at international events is certainly a valid decision for any of a number of reasons.
So is preferring to spend more time on coding than managing/mentoring/etc.
But it all has consequences and some branches will lead to more promotions/money/etc. than others. And you may be perfectly fine with that. But go into with eyes wide open.
> But you can only add so much value to an organization typing on a keyboard.
In my understanding, non-junior software development jobs never were about typing on the keyboard. Senior software engineer is a fancy name for a problem solver, and code is just a specialized tool they can build to possibly achieve the goal. It always was about talking to stakeholders, figuring out what the heck they actually want today, how it fits with what they think they want tomorrow, learning more about those stakeholders so you can guess what they will think they want next week. Only then it's thinking about it all it for a while, and only after that it's getting to press the actual buttons.
But I'm not sure those things require "soft skills" aka - in my understanding - being a people person. For me, it was a very simple learning process - I (as a junior) coded something, a manager came next month and said I have to rewrite everything again because things have changed. I hated it, so I started to think how to possibly avoid or minimize it and optimize my own processes.
And in my mental model, it's not about people (save for tiny companies where a whole department/role is a single person, so I have to account for their mental chaos monkeys), it's all about business. That's why I wrote "stakeholders", intentionally dehumanizing (with no negative connotations) the model.
Thinking about engineering leads I know who aren't about leading huge teams, it's still about mentoring, talking to people, making connections, often talking to external audiences, etc.
I think a lot of that is "soft skills." Maybe not becoming a stereotypical sales person. But it's also not don't ever bug me and let me code.
The way that most tech companies define levels - yes I’m simplifying slightly. I’ll provide citations:
Junior - you are told what to do (business objective) and how to do it (technical).
Mid - you are told what to do (business objective). But you are expected to use your experience to figure out the “how”. You should be able to lead a decently complicated feature/epic/work stream either by yourself or with others and mentoring other juniors.
Senior - You are expected to lead major projects that involve multiple epics with multiple developers, talk to “the business”, disambiguate, deal with XYProblems, communicate trade offs between time, cost, meeting requirements, etc. Now you also start having to deal with cross team coordination.
Staff - cross team impact, dealing much more with business strategy and setting technical direction.
As an IC, the only things you have at your disposal are your relationships and reputation. Both require soft skills.
I saw this at the big corporate (not faang/tech) place I work at. Engineers run and score interviews, but we don't make the final decision. That goes to HR and the hiring manager who usually has no technically background.
Yup, I have seen some really poor decisions as a result of this. I'm also curious - what will be the effect of AI assistance during behavior interviews, etc.
HR are experts in HR, which is to say they have a broader view of the institutional needs and legal requirements of hiring staffing than you do. It's always annoying when that clashes with your vision, but dismissing their entire domain is unlikely to help you avoid running into that dynamic again and again
I've never seen hiring completely in the domain of HR. HR filters incoming candidates and checks for culture fit etc, but technical competency is checked by engineers/ML folks. I can't imagine an HR person checking if someone understands neural networks.
HR involvement is unavoidable at big companies; and basics like "years of experience for payband" can cause issues.
They fundamentally do not understand the job, but somehow have to ensure its not a biased hiring process.
Yes, and it is kind of necessary when hiring people from outside trusted networks. HR makes sure if people are who they say they are, background checks, and so on. Years of experience and so on are crude filters and should be bypassable by the team/hiring manager if the candidate meets the requirements. I know in large companies this can become political.
> Company A wants to hire an engineer, an AI could solve all their tech interview questions, so why not hire that AI instead?
Interview coding questions aren't like the day-to-day job, because of the nature of an interview.
In an hour-long interview, I have to be able to state the problem in a way the candidate can understand, within 10 minutes or so. We don't have time for a lecture on the intricacies of voucher calculation and global sales tax law.
It also has to be a problem that's solvable within about 40 minutes.
The problem needs to test the candidate meets the company's hiring bar - while also having enough nuance that there's an opportunity for absolutely great candidates to impress me.
And the problem has to be possible to state unambiguously. Can't have a candidate solving the problem, but failing the interview because there was a secret requirement and they failed to read my mind.
And of course, if we're doing it in person on a whiteboard (do people do that these days?) it has to be solvable without any reference to documentation.
> In an hour-long interview, I have to be able to state the problem in a way the candidate can understand, within 10 minutes or so. We don't have time for a lecture on the intricacies of voucher calculation and global sales tax law.
If you send me a rubric I can pre-load whatever you want to talk about. If you tell me what you're trying to build and what you need help with, I can show up with a game plan.
You need to make time for a conversation on the intricacies of voucher calculation and global sales tax law if you want to find people jazzed about the problem space.
> In an hour-long interview, I have to be able to state the problem in a way the candidate can understand, within 10 minutes or so. We don't have time for a lecture on the intricacies of voucher calculation and global sales tax law.
Proving if they are technically capable of a job seems rather silly. Look at their resume, look at their online works, ask them questions about it. Use probing questions to understand the depths of their knowledge. I don't get why we are over-engineering interviews. If I have 10+ years of experience with some proof through chatting that I am, in fact, a professional software engineer, isn't that enough?
No, it's not enough. There are people out there who can talk great talk, and have great resume, but cannot do their actual job for some reason. Maybe they cannot read the code, maybe they cannot write the code, maybe they can write the code but not in the manner that keeps the rest of codebase working... I've had people like that on my team, it was miserable for all of us.
It is essential to see candidate actually write and debug code. It would be even better if we could see how the candidate deals with existing huge codebase, but sadly this kind of thing can't be easily done in a quick interview, and good candidates don't want trial periods.
>Interview coding questions aren't like the day-to-day job, because of the nature of an interview.
You have missed his point. If the interview questions are such that AI can solve them, they are the wrong questions being asked, by definition. Unless that company is trying to hire a robot, of course.
One of the best interviews I've encountered as a candidate wasn't exactly a pair programming session but it was similar. The interviewer pulled up a webpage of theirs and showed me a problem with it, and then asked how I would approach fixing it. We worked our way through many parts of their stack and while it was me driving most of the way we ended up having a number of interesting conversations that cropped up organically at various points. It was scheduled for an hour and the time actually flew by.
I felt like I got a good sense of what he would be like to work with and he got to see how I approached various problems. It avoided the live coding problems of needing to remember a bunch of syntax trivia on the spot and having to focus on a quick small solution, rather than a large scalable one that you need more often for actual work problems.
“Real problems” aren’t something that can be effectively discussed in the time span of an interview, so companies concoct unreal problems that are meant to be good indicators.
On that, these unreal questions/problems are decent proxies for general knowledge for humans, but not for AI. Humans don't have encyclopedic knowledge, so questions on a topic can do a decent job of indicating a person has the broader depth of knowledge in that topic and could bring that to bear in a job. An AI can answer all the questions but can't bring that to bear in a job.
WE saw this last year with all the "AI can now pass the bar exam" articles, but that doesn't lead to them being able to do anything approaching practicing law, because AI failure modes are not the same as humans and can't be tested the same way.
Really? How short are your interviews, and how big are these Real Problems such that you can't get a sense of how your candidate would start to tackle them?
The “real problems” most companies want people to help solve involve the evolution of products that last for years, involve repeated design discussions, in depth research, and applying retrospective learning. I don’t need someone that can just glue a Rails API together. If I did, I can literally just download that from the internet for free.
If my problems could be solved in the time span of an interview, why would I waste my time doing that interview instead of just solving it?
I don't see the issue here. Nobody expects candidates to build actual product during the interview. Having a (targeted, scope and time-limited) design discussion or giving your candidate some made-up context around an engineering cycle and then doing a retrospective with them are practical and useful ways to interview a candidate.
I'm also not sure what the alternative is? Just not hiring?
> Having a (targeted, scope and time-limited) design discussion or giving your candidate some made-up context around an engineering cycle and then doing a retrospective with them
You just described a contrived, “unreal” problem.
> I'm also not sure what the alternative is? Just not hiring?
The alternative is to come up with questions that are representative of skills related to “real problems”, as you just did, and use those instead. Unfortunately candidates consistently complain that such questions aren’t realistic.
I've tried this, but it becomes very hard to justify, with clarity, why it's a yes or no in the feedback, in a way that can be understood well as it passes through all of those up the chain that are involved with hiring.
And, I've also had people speak very well, doing great with the verbal explanation and questions, even good pseudo code, and then be unable to write a simple for loop, of any kind, in any language. These people also often have a resume full of short runs.
So, I structure mine around a, fixed, work related problem that lets me clearly justify the yes/no in a way that upper management can stomach, but then just bias my feedback a bit based on the "personal interpretation" things like what you describe (which I think are usually better indicators).
Also, resumes are 90% fiction, from what I've seen, especially from certain demographics (not allowed to perceive that though). I don't bother believing them or talking about them, unless there's time after.
Yeah, what you want is a General Intelligence that has learned the topic you care about. Google search returning an algorithm when you ask it doesn't mean that you shouldn't test candidates on that algorithm, since you still need a General Intelligence that knows it and not just the algorithm itself.
Tech interviews in general need to be overhauled, and if they were it’d be less likely that AI would be as helpful in the process to begin with (at least for LLMs in their current state).
Current LLMs can do some basic coding and stitch it together to form cool programs, but it struggles at good design work that scales. Design-focused interviews paired with soft-skill-focus is a better measure of how a dev will be in the workplace in general. Yet, most interviews are just “if you can solve this esoteric problem we don’t use at all at work, you are hired”. I’d take a bad solution with a good design over a good solution with a bad design any day, because the former is always easier to refactor and iterate on.
AI is not really good at that yet; it’s trained on a lot of public data that skews towards worse designs. It’s also not all that great at behaving like a human during code reviews; it agrees too much, is overly verbose, it hallucinates, etc.
I want to hire people who can be given some problem and will go off and work on it and come to me with questions when specs are unclear or there's some weird thing that cropped up. AI is 100% not that. You have to watch it like a 15 year old driver.
A company wants to hire someone to perform tasks X, Y and Z. It's difficult to cleanly evaluate someone's ability to do these tasks in a short amount of time, so they do their best to construct a task A which is easy to test, and such that most people who can do A can also do X, Y and Z.
Now someone comes along and builds a machine that can do A. It turns out that while for humans, A was a good indicator of X, Y and Z, for the machine it is not. A is easy for the machine, but X, Y and Z are still difficult.
This isn't a sign that the company was wrong to ask A, nor is it a sign that they could just hire the machine.
Schmidt et al., 2016 The Validity and Utility of Selection Methods in Personnel Psychology: Practical and Theoretical Implications of 100 Years of Research Findings
No, this is a message board canard. IQ tests are used at a variety of large companies with deep pockets for discrimination settlements. If there were real legal risks, that wouldn't be the case.
There are real risks for companies without deep pockets (for settlements or public relations). People I know, responsible for hiring, have told me they won't use IQ tests because of how it would come across, so the concern at least exists but how widespread is the question.
Your citation addresses it. Less than 1% of employment lawsuits are about selection criteria and employers win over 90% of them. They suggest GMA tests are _more_ defensible than other approaches.
The most interesting thing in that paper is that years of experience performed so poorly. It’s in the lowest cohort. Worse than “interests” or more general “biographical data”.
There is still the reputational risk of using selection methods with widely known disparate outcomes. Other methods also have disparate outcomes, but most of the criticism is directed at IQ tests. I've heard "IQ tests are culturally biased" but never "work sample tests are culturally biased", and I'll guess that's the experience of most hiring managers too.
Have you ever heard or read a newspaper article, or can you cite an actual example of any company actually suffering reputation harm for administering iq tests? Your citation suggests candidates view GMA tests _favorably_.
Most hiring managers believe experience matters in hiring as well, perhaps that’s the belief that keeps them from using iq tests.
For what it’s worth, IQ tests are biased (see duyme’ adoptive studies for a drastic economic impact). That largely is orthogonal to if they are predictive in the ways your citation outlines.
Somehow these reputational risks accrue to the median hiring company, but not to the global brands like PepsiCo and Proctor & Gamble that do GMA testing already. I maintain: this is a message board trope.
McK's case interview is just as game-able as HackerRank style interviews. There are entire consulting clubs at many colleges that teach this exact interview style. It's true that it's harder (but not impossible) to use AI to help, but calling it an IQ-like test is true only as much as any other technical interview.
That being said, McK did create an entire game that they claim can't be studied for ahead of time. If the intention is to test true problem solving skills, then maybe that's roughly equivalent to a systems interview, which is hard(er) to cheat .
> That being said, McK did create an entire game that they claim can't be studied for ahead of time. If the intention is to test true problem solving skills, then maybe that's roughly equivalent to a systems interview, which is hard(er) to cheat .
Maybe, but they have hundreds! And new ones every quarter! Having done both systems and case interviews I can say that leaking is not as big a problem as with coding interviews. Sure you can memorize how to build Netflix/Zoom, or how to analyze P&L for a sandwich shop, but those interviews depend on interviewers throwing in complications or asking clarifying questions.
This is a great point. Though what if the answer is that the company can hire that AI to solve a significant fraction of its actual problems? People who do the assessments and decide what features should look like are often called managers (product, engineering, etc.).
For a while I’ve been skeptical that the rate of hiring of engineers would change significantly because of LLMs, but I’m starting to feel like maybe I’m wrong and it’s already changing and companies are looking toward AI to lower costs and require fewer humans. In that case they are probably still going to want people who are technically exceptional - maybe even more so - but are able and willing to create, integrate, and babysit AI generated code, and also do PM and EM style feature management.
If companies are slowing hiring due to AI, I would expect interviews to get worse before they get better.
> For example, it could be that the company wants the engineer to do some kind of assessment whether a feature should be implemented at all, and if yes, in what way. Then you could, in an interview, give a bit of context and then ask the candidate to think out loud about an example feature request.
In most companies every engineer above a junior level is expected to pass features and bugfixes through their common sense filter and provide feedback. Product managers and designers aren't infallible and sometimes lack knowledge about the system or product that an engineer might have.
You can't just take requirements and churn out code without a critical eye at what you're doing.
Maybe now, or maybe in a year or two, AI coding tools will be good enough that a single semi-technical person can be Product Manager for a small product, and implement all the feature through AI/LLM tools.
Probably not for something of the complexity of Google Maps, but for a simpler website with some interactive elements, that could work.
But then, this was just an example. There can be lots of reasons that companies still need engineers, my point was that they need to think about these reasons, and then use these reasons to decide how to select their engineers.
I was asked by an SME to code on a whiteboard for an interview (in 2005? I think?). I asked if I could have a computer, they said no. I asked if I would be using a whiteboard during my day-to-day. They said no. I asked why they used whiteboards, they said they were mimicking Google's best practice. That discussion went on for a good few minutes and by the end of it I was teetering on leaving because the fit wasn't good.
I agreed to do it as long as they understood that I felt it was a terrible way of assessing someone's ability to code. I was allowed to use any programming language because they knew them all (allegedly).
The solution was a pretty obvious bit-shift. So I wrote memory registers up on the board and did it in Motorola 68000 Assembler (because I had been doing a lot of it around that time), halfway through they stopped me and I said I'd be happy to do it again if they gave me a computer.
I work for a faang subsidiary. We pay well below average salary and equity. We finally got one nice perk, a very good 401k match. A few months later it was announced that the 401k match would be scaled back "to come in line with what our parent company offers". I thought about asking "will be getting salaries or equity in line with what our parent company offers?" but that would have been useless. Management doesn't care. I'm job hunting.
> I was asked by an SME to code on a whiteboard for an interview (in 2005? I think?). I asked if I could have a computer, they said no. I asked if I would be using a whiteboard during my day-to-day. They said no. I asked why they used whiteboards, they said they were mimicking Google's best practice.
This looks more like a culture fit test than a coding test.
Folks getting mad about whiteboard interviews is a meme at this point. It misses the point. We CANT test you effectively on your programming skillbase. So we test on a more relevant job skill, like can you have a real conversation (with a whiteboard to help) about how to solve the problem.
It isn't that your interviewer knew all the languages, but that the language didn't matter.
I didn't get this until I was giving interviews. The instructions on how to give them are pretty clear. The goal isn't to "solve the puzzle" but instead to demonstrate you can reason about it effectively, communicate your knowledge and communicate as part of problem solving.
I know many interviewers also didn't get it, and it became just "do you know the trick to my puzzle". That pattern of failure is a good reason to deprecate white board interviews, not "I don't write on a whiteboard when i program in real life".
> We CANT test you effectively on your programming skillbase. So we test on a more relevant job skill, like can you have a real conversation (with a whiteboard to help) about how to solve the problem.
Except, that's not what happens. In basically every coding interview in my life, it's been a gauntlet: code this leetcode medium/hard problem while singing and tapdancing backwards. Screw up in any way -- or worse (and also commonly) miss the obscure trick that brings the solution to the next level of algorithmic complexity -- and your interview day is over. And it's only gotten worse over time, in that nowadays, interviewers start with the leetcode medium as the "warmup exercise". That's nuts.
It's not a one off. The people doing these interviews either don't know what they're supposed to be looking for, or they're at a big tech company and their mandate is to be a severe winnowing function.
> It isn't that your interviewer knew all the languages, but that the language didn't matter.
I've done enough programming interviews to know that using even a marginally exotic language (like, say, Ruby) will drastically reduce your success rate. You either use a language that your interviewer knows well, or you're adding a level of friction that will hurt you. Interviewers love to say that language doesn't matter, but in practice, if they can't know that you're not making up the syntax, then it dials up the skepticism level.
They generally do not know what they are looking for. They are generally untrained, and if they are trained, the training is probably all about using leetcode-type problems to give out interviews that are sufficiently similar that you can run stats on the results and call them "objective", which is exactly the thing we are all quite correctly complaining about. Which is perhaps anti-training.
The problem is that the business side wants to reduce it to an objective checklist, but you can't do that because of Goodhart's Law [1]. AI is throwing this problem into focus because it is basically capable of passing any objective checklist, with just a bit of human driving [2]. Interviews can not consist of "I'm going to ask a question and if you give me the objectively correct answer you get a point and if you do not give the objectively correct answer you do not". The risk of hiring someone who could give the objectively correct answers but couldn't program their way out of a wet paper bag, let alone do requirements elicitation in collaboration with other humans or architecture or risk analysis or any of the many other things that a real engineering job consists of, was already pretty high before AI.
But if interviewing is not a matter of saying the objectively correct things, a lot of people at all levels are just incapable of handling it after that. The Western philosophical mindset doesn't handle this sort of thing very well.
[2]: Note this is not necessarily bad because "AI bad!", but, if all the human on the other end can offer me is that they can drive the AI, I don't need them. I can do it myself and/or hire any number of other such people. You need to bring something to the job other than the ability to drive an AI and you need to demonstrate whatever that is in the interview process. I can type what you tell me into a computer and then fail to comprehend the answer it gives is not a value-add.
It is a gross oversimplification but you can look at the Western mindset as being a reductionistic, "things are composed of their parts" sort of view, and the Eastern mindset as a holistic mindset where breaking things into their components also destroys the thing in the process.
The reality isn't so much "in between" as "both". There is a reason the West developed a lot of tech and the East, despite thousands of years of opportunity, didn't so much. But there is also a limit to the reductionistic viewpoint.
In this case, being told that the only way to hire a truly good developer is to make a holistic evaluation of a candidate, that you can not "reduce" it to a checklist because the very act of reducing it to a checklist invalidates the process, is something that a lot of Western sorts of people just can't process. How can something be effectively impossible to break into parts?
On the other hand, it is arguably a Western viewpoint that leads to the idea of Goodhart's law in the first place; the Eastern viewpoint tends to just say "things can't be reduced" and stop the investigation there.
This is highly stereotypical, of course, and should be considered as an extremely broad classification of types of philosophy, and not really associated directly with any individual humans who may happen to be physically located in the east or west. Further as I said I think the "correct" answer is neither one, nor the other, nor anything in between, but both, so I am not casting any shade on any country or culture per se. It is a useful, if broad, framework to understand things at a very, very high level.
When I joined my current team I found they had changed the technical test after I had interviewed but before I joined. A couple of friends also applied and got rejected because of this new test.
When I finally got in the door and joined the hiring effort I was appalled to find they’d implemented a leetcode-esque series of challenges with criteria such as “if the candidate doesn’t immediately identify and then use a stack then fail interview”. There were 7 more like this with increasingly harsh criteria.
Do I? yes. I also teach my students that the goal of an interview is to convince the interviewer you are a good candidate, not to answer the questions correctly. Sometimes they correlate. Give the customer what they need not what they asked for.
Do I see others doing so? sadly no.
I feel like a lot of the replies to my comment didn't read to the end, I agree the implementation is bad. The whiteboard just isn't actually the problem. The interviewers are.
Unless they change mentality to "did this candidate show me the skills i am looking for" instead of "did they solve puzzle" the method doesn't matter.
The replies are addressing the reality of the interview landscape that fails to live up to your theory of how whiteboarding interviews should be.
It's all well and good that you and other "wise interviewer" commenters on HN actually grok what the point of interviews are, but you are unicorns in the landscape.
I don't think you made it to the last paragraph either:
> I know many interviewers also didn't get it, and it became just "do you know the trick to my puzzle". That pattern of failure is a good reason to deprecate white board interviews, not "I don't write on a whiteboard when i program in real life".
> The goal isn't to "solve the puzzle" but instead to demonstrate you can reason about it effectively, communicate your knowledge and communicate as part of problem solving.
...while being closely monitored in a high-stakes performance in front of an audience of strangers judging them critically.
> Why are you on a thread about Google-style interviews?
For the same reason you wrote "Google-style". Because this thread is specifically about those interviews happening not at Google.
Oh, maybe you misunderstood their question. When they suggested Google wasn't relevant, they meant the company culture at Google itself because that's what you were talking about.
Perhaps. I'd even say it's part of what is taught as part of a PhD.
But if someone was ready for your exact question by having the right interview practice/experience, or they just don't care about your job so there's no stakes. Then you still aren't measuring what you think you are.
> So we test on a more relevant job skill, like can you have a real conversation (with a whiteboard to help) about how to solve the problem.
Everybody says that, but reality is they don't imho. If you don't pass the pet question quiz "they don't know how to program" or are a "faker", etc.
I've seen this over and over and if you want to test a real conversation you can ask about their experience. (I realize the challenge with that is young interviewers aren't able to do that very well with more experienced people.)
+1 to all this. It still surprises me how many people, even after being in the industry for years, think the goal of any interview is to “write the best code” or “get the right answer”.
What I want to know from an interview is if you can be presented an abstract problem and collaboratively work with others on it. After that, getting the “right” answer to my contrived interview question is barely even icing on the cake.
If you complain about having to have a discussion about how to solve the problem, I no longer care about actually solving the problem, because you’ve already failed the test.
I think you're severely underestimating how much just about every software company has bought into the FAANG philosophy, and how many candidates they get who can answer those questions correctly.
Yes if you don't communicate clearly, you will get points deducted. But if you can't answer the question nearly perfectly, its basically an immediate fail.
Unfortunately I used to think this was the main purpose of the interview as well, but have been proven wrong time and time again.
The only thing that matters in most places is getting to the optimal solution quickly. It doesn't matter if you explain your thought process or ask clarifying questions, just get to the solution and answer the time and space complexity correctly and you pass.
Like others have said I think this is a symptom of the sheer number of people applying and needing to go through the process, there is no time for nuance or evaluating people on if you would actually like to work with them or not.
I am so happy that you did this. We vote with our feet and sadly, too many tech folks are unwilling to use their power or have golden handcuff tunnel vision.
>I was allowed to use any programming language because they knew them all (allegedly).
After 30 years of doing this, I find that typically the people who claim to know a lot often know very little. They're insecure in their ability so much that they've tricked themselves into not learning anything.
What is the functional difference between copying an AI answer and copying a StackOverflow answer, in terms of it being "cheating" during an interview?
I think the entire question is missing the forest for the trees. I have never asked a candidate to write code in any fashion during an interview. I talk to them. I ask them how they would solve problems, chase down bugs, or implement new features. I ask about concepts like OOP. I ask about what they've worked on previously, what they found interesting, what they found frustrating, etc.
Languages are largely teachable, it's just syntax and keywords. What I can't teach people is how to think like programmers need to: how to break down big, hard problems into smaller problems and implement solutions. If you know that, I can teach you fucking Swift, it isn't THAT complicated and there's about 5 million examples of "how do I $X" available all over the Internet.
> Languages are largely teachable, it's just syntax and keywords.
This is like "learning a natural language is just 'cramming vocabulary and grammar' - voila, you've become a fluent C1 speaker". :-)
Seriously: if you argue this way, you have only seen a very biased set of programming languages, and additionally, your knowledge of these programming languages is very superficial (i.e. you have never gotten to the "interesting"/"deep" concepts that make this particular programming language special, and which are hard to replicate in most other programming languages).
I think the argument is that for a good chunk of business work, you don't need to use the "interesting"/"deep" concepts. Sure, you'll need time to adapt to the idioms of the language you're using, but following examples you can be just as productive as others in a relatively short time.
But companies don't pay high salaries for the 80% mundane and easy tasks you do day to day. They pay for the 20% that is challenging and especially for that 1% of problems that occur only once every few months or years. If that 80% was 100% of the job then the company could pay 1/2 to 1/3rd the amount by outsourcing it.
I disagree, companies pay based on the problems you can solve to make them money or help achieve organizational goals. One of those ways can be coding, but there are many others.
In C, implicit type narrowing/widening behavior stumped me as a noob working on noob problems. “Deep C Secrets” was a revelation when I finally found it.
No, the comparison to natural languages is what is whack. If you understand the underlying concepts that programming languages pick and choose from as features, all you have to learn is what keywords map to those concepts and the language's syntax.
The comparison to natural languages would be if you could learn one language and then quickly pick up every other language of that "class" after learning how that single language works. That's not really how natural language works at all, but it does work with programming languages.
> If you understand the underlying concepts that programming languages pick and choose from as features, all you have to learn is what keywords map to those concepts and the language's syntax.
If you understand the grammatical topics that a natural language picks, all you have to learn is what word transformation rules map to those concepts, and the natural language's vocabulary.
> The comparison to natural languages would be if you could learn one language and then quickly pick up every other language of that "class" after learning how that single language works.
There do exist books on this topic (though more commonly for language families). See for example
> If you understand the grammatical topics that a natural language picks, all you have to learn is what word transformation rules map to those concepts, and the natural language's vocabulary.
Yes, and then do that in real time while you're having a conversation with someone who's been learning the language since they were a baby. It is an unreasonable comparison.
> Languages are largely teachable, it's just syntax and keywords.
That's only true for a subset of programming languages, and it requires you to already know how to program in at least another language of the same family. Knowing Java will not help you with Haskell, but it will help you with C#.
I have to deal with students using AI to cheat on homework and exams, and I can't allow them to not even learn the basic concepts.
They could convince you with buzzwords, get hired, and then feed all problems to the AI until it reaches a point where the codebase is too big for the context, and then all their prompt “engineering” experience is basically useless.
That is the future I am trying to prevent.
Until the AI can code a full system like SAP, or an Operating System, or a Photoshop clone, by itself, we need some people in the loop, and the more knowledgeable the people, the better.
> That's only true for a subset of programming languages, and it requires you to already know how to program in at least another language of the same family. Knowing Java will not help you with Haskell, but it will help you with C#.
In this context, to a large extent it holds. Yeah. It’s probably more true of mainstream languages roughly related to each other, but in an interview, you’re not testing for knowledge of syntax and keywords. You’re trying to test for ability to build and problem solve.
I share your concern about prompt “engineers” who don’t understand the underlying codebase, language or system. But I don’t know what to do about it in the context of a technical interview.
It's not useless because some people will lie and cheat. Over the years, I've interviewed hundreds of people and a substantial minority could not write even the simplest code. Many will say they would be able to filter out such people after a conversation. But, IMO, the fact that they are still able to get hired at some places shows how wrong that often is.
I've accidentally been using an AI-proof hiring technique for about 20 years: ask a junior developer to bring code with them and ask them to explain it verbally. You can then talk about what they would change, how they would change it, what they would do differently, if they've used patterns (on purpose or by accident) what the benefits/drawbacks are etc. If they're a senior dev, we give them - on the day - a small but humorously-nasty chunk of code and ask them to reason through it live.
Works really well and it mimics the what we find is the most important bit about coding.
I don't mind if they use AI to shortcut the boring stuff in the day-to-day, as long as they can think critically about the result.
Yep. I've also been using an AI-proof interview for years. We have a normal conversation, they talk about their work, and I do a short round of well-tested technical questions (there's no trivia, let's just talk about some concepts you probably encounter fairly regularly given your areas of expertise).
You can tell who's trying to use AI live. They're clearly reading, and they don't understand the content of their answers, and they never say "I don't know." So if you ask a followup or even "are you sure" they start to panic. It's really obvious.
Maybe this is only a real problem for the teams that offloading their interviewing skills onto some leetcode nonsense...
This is a fine way. I’ll say that the difference between a senior and a principal is that the senior might snicker but the principal knows that there’s a chance the code was written by a founder.
And if the Principal is good, they should stand up and say exactly why the code is bad. If there's a reason to laugh because it is cliche bad, they should say so.
If someone gave me code with
if (x = 7) { ... } as part of a C eval.
Yeah, you'll get a sarcastic response back because I know it is testing code.
What I think people ignore is that personality matters. Especially at the higher levels. If you are a Principal SWE you have to be able to stand up to a CEO and say "No, sir. I think you are wrong. This is why." In a diplomatic way. Or sometimes. Less than diplomatic, depending on the CEO.
One manager that hired me was trying to figure me out. So he said (and I think he was honest at the time). "You got the job as long an you aren't an axe murderer."
To which I replied deadpan: "I hope I hid the axe well." (To be clear to all reading, I have never killed someone, nevermind with an axe! Hi FBI, NSA, CIA and pals!)
Got the job, and we got along great, I operated as his right hand.
> Using apps like GitHub Co-pilot and Cursor to auto-complete code requires very little skill in hands-on coding.
this is a crazy take in the context of coding interviews. first, because it's quite obvious if someone is blindly copy and pasting from cursor, for example, and figuring out what to do is a significant portion of the battle, if you can get cursor to solve a complex problem, elegantly, and in one try, the likelihood that you're actually a good engineer is quite high.
if you're solving a tightly scoped and precise problem, like most coding interviews, the challenge largely lies in identifying the right solution and debugging when it's not right. if you're conducting an interview, you're also likely asking someone to walk through their solution, so it's obvious if they don't understand what they're doing.
cursor and copilot don't solve for that, they make it much easier to write code quickly, once you know what you're doing.
Nowadays I am on the other part of the fence, I am the interviewer. We are not a FAANG, so we just use a SANE interview process. Single interview, we ask the candidate about his CV and what his expectations are, what are his competences and we ask him to show us some code he has written. That's all. The process is fast and extremely effective. You can discriminate week candidates in minutes.
That process might work for your company precisely because you are not FAANG. You don't get hundreds of applicants that are dying to get in, so people don't have that strong of a motivation to do anything it takes (including lying) to get the job.
I’ve worked at a company with 150,000 employees. The interview process was pretty much as described here. There is absolutely no reason a Big Co needs to operate any differently.
How do you expect them to get access to the property internal Git repo codebase and approval from their employer's lawyers to show it to third parties during the interview?
Sounds like you're only selecting Foss devs and nothing more.
Most people have still written code for school or a hobby project. Maybe I'm missing empathy, but I cannot understand how some developers have no code to show.
If that's the case however, just let them make a small project over the weekend and then do another interview where you ask stuff about what they've made. It's not that deep
> Most people have still written code for school or a hobby project. Maybe I'm missing empathy, but I cannot understand how some developers have no code to show.
First: they might have private code, but not necessarily code to show (I, for example, am rather not willing to show quite some of the code that I wrote privately).
Second: the kind of "code" that I tend to write privately (and into which I invest quite a lot of time) is really different from what I do at work, and what is actually considered "code" by many. It's more like (very incomplete) drawings and TeX notes about observations and proofs of properties and symmetries between some algorithms. Once finished, they will be very easy to systematically transform into a program in a computer language.
This is about very novel stuff, which to explain would take quite a lot of time.
The objective in an interview like this shouldn't be to grade the quality of the code you bring in any sort of scale, but to have a discussion about the options you took. In that sense, it really matters very little what you present as long as we can do a small back-and-forth that lets me into what sort of person you are.
> but to have a discussion about the options you took
I can clearly state that this is not I commonly think about code that I write privately (and also for code that I write for the job only if I must). For private code, I rather commonly start with a "gut feeling" about some unexpected symmetry that the problem that I am working on likely has, then try to formulate these "gut feelings" as mathematical properties, and later theorems. At the end, everything "fits (for outsiders: unexpectly) together".
Thus, there is hardly ever a "option that I took", but rather a "I let everything flow: from the source [my gut feeling] to the sea [which is - ironically - the source (code)]".
> Most people have still written code for school or a hobby project
School was years and years ago, and has nothing to do with my current skills.
From the people i personally know, most do _not_ have a hobby project, even fewer have hobby projects that showcases their technical skills. Nor should they be expected to. Most people have non-programming hobbies.
> I cannot understand how some developers have no code to show.
It's really not that deep, I'm worried if you really cannot understand. I don't code outside of work, I'm not interested in doing it. I'm good at software engineering, not passionate about it. I have a bunch of other hobbies. There's no reason I'd have any code to show now or at any point in the future.
> let them make a small project over the weekend and then do another interview where you ask stuff about what they've made
If I'm paid for it, sure why not I could do that. I won't love it but hey I'm looking for a job, I'll put the legwork in.
But if this is the only or the "preferred" interview process for a company, I need to point out that it is deeply discriminatory as it advantages people who have the time to do a weekend project: for example it benefits males disproportionally (women do most of the care work in any country, also the most house work, also have a higher chance to be a single parent, all of which impacts the time they can put in a "weekend project" if they can do it at all).
> It's really not that deep, I'm worried if you really cannot understand. I don't code outside of work, I'm not interested in doing it. I'm good at software engineering, not passionate about it. I have a bunch of other hobbies. There's no reason I'd have any code to show now or at any point in the future.
It's like asking a dentist interview candidate to show you examples of fillings and crowns they did at home as a hobby. I don't understand why there is this automatic assumption that people who program at work also do it outside of work.
I ask for code, and if they have none prepared I ask that they spend at most two hours building something they enjoy.
I expect a decent developer to be able to bootstrap and write most of a fun toy project in a domain they know well or at worst some kata from the Internet within half an hour. Then we spend some time screen sharing and talking about it, similar to pair programming but less problem focused.
If you can't do it you'll likely struggle a lot when working with us because we commonly use throwaway prototypes.
Sure. A bigger organisation with more layers in their technology department could treat candidates better that would struggle with a hiring process like mine. More detailed tickets, more mentors available, bigger cashflow, things like that.
In such settings it's also somewhat common to hire consultants in bulk, like 5-10 at a time, try them out for six months and keep the ones that enjoy the work and fit well in the organisation, and over time try to employ some of them directly.
I realize it's a rhetorical question, but pretty much everybody in the arts? I'd be very surprised to find a professional musician that doesn't play as a hobby.
Artists and musicians surely have portfolios to show, but I don't think those are usually composed of things they're doing in their free time. Maybe? I'm not an artist so maybe I'm wrong. I guess then we're back to "you should have a portfolio" rather than specifically "you should do your job in your free time." I can agree with that!
If you're not interested, you're not interested. Not even about "passion" at that point, but the bare minimum interest in your industry. I said it better previously: https://news.ycombinator.com/item?id=15553482
I don't code much outside of work. I have hobby projects from 10+ years ago, but they're not much more than landing pages copied from templates and wordpress installs. I mostly work in backend/data/platform engineering professionally.
If I were asked to make a small project over a weekend, I'd be likely to decline rather than doing a more standard interview, or I'd use AI to do it in a reasonable timeframe (which seems to defeat the purpose as it relates to this discussion)
This points to another issue - when I do code outside of work, it's often specifically to try out things I don't do at work. After a day of doing backend work, I'll maybe put together a basic web UI for something. That code is likely awful because I just need it to be functional more than good, and also probably not related to the work I'd be hired to do.
My most recent real "side projects" are a terrible OSS monte carlo simulator tool that I contributed to, but cannot explain most of the code for, and a half-working React application that has performance issues I never fixed. Both are years old at this point. I'm not sure what an interviewer would gain from those.
This relates to another issue of using people's public github as a hiring signal. I don't share any of these repos because at a glance the code is ugly, broken, incomplete.
Below the surface, I'm probably scratching a very interesting itch. Exploring a specific idea or problem, and then I stop when I get my answer.
I started writing code when I was 12 and started doing it professionally at 22. I'm now in my mid-30s and outside of work, I haven't written anything more than one-off scripts for my homelab in close to a decade. I'm already spending upwards of 50 hours with code each week and I need to do something else at night and on the weekends to release my brain from it. I also didn't go to school for CS, and even if I did... it was over a decade ago. So I have ~25 years of experience writing code but could not show you a single line of it. And even if I could, how would you know I was the one to write it?
This is an extremely flawed interview process in my opinion and the last time I encountered it led to an awkward scenario that led to me walking out. Personally, when I conduct interviews, it's a mix of things. We talk about your past work, I quiz you a bit on some topics you'd encounter in your day-to-day here, and then we'll spend an hour doing some combination of a code review of a working-but-flawed demo project I created, a 30-40 minute coding exercise, and/or a problem-solving scenario where I give you a problem and then we talk through how, as a pair, how we could solve it.
School was a few decades ago, and the code I have on Github is mostly toy stuff I do in rainy weekends, most of us have a life without room to code outside work most of the time.
Like many of the other commenters, I have no code to show. I'm strongly motivated at work to solve problems and create correct, performant, maintainable code. I appreciate a job well done.
Outside of work, I just don't have the motivation to code anything. I don't have sufficient at-home problems where code will fix them.
In an interview, ask me anything! ... except to show you code on Github.
I’ve been working professionally for almost 30 years. I have never written a single line of code “for fun”. I write code for money. I then take that money to fund my hobbies. The absolutely last thing I want to do when I get off work is stare at a computer.
If I already have a job, unless you are paying top of market, why would I spend my weekend writing code?
It looks like you professionally sold 30 years of your life for money with no fun. You could have done something for fun all this time, and got payed for it, too. Much better that way.
I did no such thing, I spent the first 15 years as a part time fitness instructor mostly for the social aspect, hanging out with friends and training for and doing group runs and a little travel.
I spent the next 8 years married (still married) and raising two step sons and spent the last two and half years traveling extensively including over a year doing the “digital nomad thing”.
We have been averaging getting on a plane to do something on average over a dozen times a year since late 2021.
Of course the Covid lockdown slowed us down for two years.
When I am at home in Florida, I go swimming at one of the multiple pools or workout at one of the two gyms that’s part of our complex. It’s warm enough most of the year at least during the day.
During the weekends, I go downstairs and hang out at the bar and just sip soda while hanging out with my friend the bartender and whoever else is down there.
I “retired my wife” in 2020 when I was 46 and she was 44 8 years into my marriage so she could pursue her passion projects and we could pick up and travel as often as we wanted to - the joys of working remotely.
Who are these "most people"? School was over 10 years ago for me when schoolwork was not posted on GitHub nor is it relevant to my current job anymore, and I don't do hobby coding since I have other hobbies and responsibilities.
WTF is this hobby coding bullshit expectations? What other professions expect you do more work after work as a hobby and show it? Do bus drivers film themselves driving busses after work as a hobby? Do surgeons cut up people in their spare time as a hobby?
It's that trimodal-comp thing. The 6-hour leetcode gauntlet is only the overwhelming norm in the top hump of that graph. It's not normal in the other two (not that it's never seen, but the companies dumb enough to do it without offering FAANG-tier comp are shooting themselves in the foot). I've had, IDK, ten plus tech jobs and have been solidly in the middle comp tier for most of those, and have exactly once encountered anything resembling a leetcode question in the wild.
I've also done hiring, and have no idea what good leetcode would do me. I'm convinced these people thinking they "dodged a bullet" on "fakers" either spotted a real faker who they would have caught with a normal conversation anyway, or else are assuming someone with a good resume who failed their test did so because they were lying and not because they choked under the unique sort of pressure interviews present when you turn them into a dancing-monkey routine (it's approximately the same kind of stress as doing an open mic night or karaoke in front of a crowd of strangers—most people have trouble with that and lots of them fall apart at least some of the times they try it). Meanwhile, anyone who can talk about the job and their career in any depth, convincingly, without giving away that they're actually either very-green or have no idea what they're doing, while in fact not knowing how to do the job, possesses a skill at least as valuable as programming, and I find it hard to believe most such folks haven't figured out they can apply that skill directly in exchange for money instead of trying to fake their way through conversational tech interviews.
I do see how leetcode is valuable if you want to ensure that most of the candidates in your pipeline would do fine before you even evaluate them, because you offer high comp and need a way to discourage candidates who definitely can't make it before they even apply, and/or if you want to make job-hopping painful as a wink-and-nudge collusion way to keep comp suppressed. It makes sense for FAANG, in a certain way, but not because it's a good way to evaluate candidates per se.
> WTF is this hobby coding bullshit expectations? What other professions expect you do more work after work as a hobby and show it? Do bus drivers film themselves driving busses after work as a hobby? Do surgeons cut up people in their spare time as a hobby?
I think programming has more commonality with other creative, 'soft' jobs like graphic design (which itself can involve programming), architecture, media, marketing, etc than meets the eye.
Many of these roles require that applicants have some sort of portfolio that can be perused by the interviewer freely. I feel co-opting that word—'portfolio'—would do us software developers a big favour instead of trivialising outside-of-work programming as 'side projects' or 'hobbies'.
>architecture, media, marketing, etc than meets the eye.
I disagree. Programing is more engineering than art. Art doesn't have source code. You can show the final painting and I can show the final product I worked on but not the source code I wrote as that belongs to my employer. Also, most art like paintings are not done by large teams, so you can show what you did in that painting but in a large SW projects, I can't show what exactly form the final product I did and what else was done by my team.
Most of my valuable work in programing is engineering, especially fixing bugs, not creating portfolios to show off. I have nothing publicly to show off, mostly because firstly, it's private to my former employers, and secondly because code gets outdated and replaced fast, most of what I worte in the past probably doesn't run today anymore, but have made my employers happy and wealthy.
Sounds like you just treat programming more like engineering than art. Some art does have source code, there is plenty of room for creative exploration with code.
>there is plenty of room for creative exploration with code.
That also pays the bills? That's not my experience. That's what hobbies are for. Jobs are for paying bills. Paying bills with hobbies an art are a luxury for privileged.
I said neither 'art' nor 'paintings' which you have fixated on for some reason. I mentioned creative endeavours that are generally team-based but all generate some sort of portfolio. Whether that portfolio is from work or done in one's personal time, it is still a portfolio of past work.
Plus, software engineering is absolutely a creative endeavour. And I daresay normal 'engineering' (civil, mechanical, aero, etc) is a creative endeavour too; it's just a matter of egos and that seem to separate STEM versus non-STEM. There are portfolios for everything. I don't understand the desire for software engineers to just waltz into an interview, claim to have done X, Y, Z, with no proof, and secure a job.
The proof part is interesting. Civil is easy to prove because of its artifacts. Someone from Netflix or Meta layoffs, what proof do any of them have? Do some people defensively maintain background proof other than paycheck stubs?
It could be considered similar to scaffolding or boilerplate in code, except usually none of this is visible in the end product, while the code boilerplate is always there. These lines are drawn light and completely covered up by the end result - sometimes even manually erased depending on the medium.
I don't like writing code for the sake of it, and have gotten a lot better, over 25 years of writing code, at evaluating whether I need to write code or whether I'd be better off using something that already exists and putting up with its limitations, or even just doing nothing (see that XKCD comic with the time-savings payoff chart).
The result is that I don't think I've written anything longer than about a ten-line shell or python or JS script for my personal use in... a decade or more.
Frankly I probably think you shouldn't be paying anyone to do the thing you're wanting to pay me to do, because computers are likely just an expensive distraction that management's pursuing because the promise of legibility, even if in-fact pointless in this case, is incredibly enticing to them, but also I like money and will build the thing you shouldn't be building for you if you pay me. I'll even do it well, if you let me. But I don't make the same mistake (much) in my own life, any more.
Would I write a bunch of code on my own if I thought it'd be worth it? Yes, but that'd almost certainly mean I had a product idea. If I were any good at thinking of product ideas, I'd long since have had my own business. I'm terrible at it. That's literally the only reason I'm applying for a job. If I had a pile of decent code to show you, it'd be because I didn't need your job.
I can empathize with your position if you are also against expecting candidates "prepare for the interview" by leet code grinding or "brush up on CS concepts".
Hobby coding is million times better than that crap.
At least leetcode is something somewhat standardized (algorithms don't change). Hobby coding is not, it's something subjective and varying between the interests of each candidates and often have nothing to do with a job.
My worst code is always what I wrote yesterday. Often what’s missing is context, unless I comment ad nauseam. Sure I didn’t write complete test, obey open closed principles abstract into factory functions. The code I send from my hobby projects is likely a mess, because finishing on my own time by my own unpaid constraints wills it to be so
Maybe you forked a library because of reasons. You can tour the original repo and explain the problems. I have at least one of those examples for each time the legal or confidentiality department stepped in.
The word maybe is doing a lot of heavy lifting here. What if you never had to do that? Not everyone's work is public. Inf act I'd say most people's work is not public. Sometimes even the product is not public since it's B-2-B.
But if you’ve worked on something mature and nontrivial, you’ve forked a dependency and are able to tour it. Looks like I’ve done it on average twice per year.
We do this too, works fine. We ask open ended questions like, "What's your favorite thing you've done in your career and why?" and "What was the most challenging project in your career and why?" If you listen, you can get a lot of insight from just those two questions. If they don't give enough detail, we'll probe a little.
Our "gotcha," which doesn't apply to most languages anymore is, "What's the difference between a function and a procedure." It's a one sentence answer, but people who didn't know it would give some pretty enlightening answers.
Edit: From the replies I can see people are a little defensive about not knowing it. Not knowing it is ok because it was a question I asked people 20 years ago relevant to a language long dead in the US. I blame the defensiveness on how FUBAR the current landscape is. Giving a nuanced answer to show your depth of knowledge is actually preferred. A once sentence answer is minimal.
I'm editing this because HN says I'm posting too fast, which is super annoying, but what can I do?
> We do this too, works fine. We ask open ended questions like, "What's your favorite thing you've done in your career and why?" and "What was the most challenging project in your career and why?" If you listen, you can get a lot of insight from just those two questions. If they don't give enough detail, we'll probe a little.
The problem is: there is a very negative incentive to give honest answers. If I were to answer these questions honestly, I'd bring up some very interesting theorems (related to some deep algorithmic topics) that I proved in my PhD thesis. Yes, I would have loved to stay in academia, but I switched to industry because of the bad job prospects in academia - this is not what
interviewers want to hear. :-(
> "What's the difference between a function and a procedure." It's a one sentence answer
The terminology here differs quite a lot in different "programming communities". For example
says: "Procedure (computer science), also termed a subroutine, function, or subprogram",
i.e. there is no difference. On the other hand, Pascal programmers strongly distinguish between functions and procedures; here functions return a value, but procedures don't. Programmers who are more attracted to type theory (think Haskell) would rather consider "procedures" to be functions returning a unit type. If you rather come from a database programming background, (stored) procedures vs functions are quite different concepts.
I could go on and on. What I want to point out is that this topic is much more subtle than a "one sentence answer".
> I'd bring up some very interesting theorems (related to some deep algorithmic topics) that I proved in my PhD thesis. [...] I switched to industry because of the bad job prospects in academia - this is not what interviewers want to hear.
In my experience you'll be fine giving that answer assuming you're going for the kind of programming job that hires PhDs.
You remind them you have a PhD - and in something deeply algorithmic. You can successfully answer any follow-up questions from them, as you literally have a PhD in the topic they're asking about. There's no shame in entering industry because you want jobs and money - in fact, those things are precisely what the hiring manager is able to offer you.
You'd rather be in academia but it doesn't have the pay and job security? Well, the hiring manager would rather be a snowboard instructor in Aspen but doesn't for the same reason. So you've got common ground with them.
> Yes, I would have loved to stay in academia, but I switched to industry because of the bad job prospects in academia - this is not what interviewers want to hear. :-(
I would love to hear that from a candidate I'm interviewing. Who can't relate to the distinction between your ideal job and the job that will actually pay you money?
>The problem is: there is a very negative incentive to give honest answers. If I were to answer these questions honestly, I'd bring up some very interesting theorems (related to some deep algorithmic topics) that I proved in my PhD thesis.
This is unfortunate that you would get that response. FWIW, I would be interested in hearing all this in an interview and I would look at it favorably.
>What I want to point out is that this topic is much more subtle than a "one sentence answer".
Yes, you would definitely get bonus points for nuance. The one sentence answer was minimal. What it filters out are people who don't know anything about Delphi but applying for the job with highly embellished resumes hoping to get lucky. This was for software used in hospitals, so bugs or errant code could have pretty drastic consequences.
Here's an interesting thought on your "gotcha" - I'm 57 years old, been programming as a career for over 30 years, a lot of languages and I have no idea what the difference is.
If I'm applying for a Java position and I claim to have Java experience on my resume, it's perfectly valid for them to ask me the difference between an int, an Integer, and a BigInteger.
But it's certainly not a universal question applicable to all programming languages.
Likewise, Clubber says in their post that their 'gotcha' question doesn't apply to most languages.
I have no idea either. I can easily look it up though. You can often tell an inexperienced interviewer from the extremely domain specific question they ask which _they_ are familiar with.
There are some absolutely ridiculous qs I've been asked like this and they've all had no followup question to illuminate why it would have been relevant
1. what version of java do you use? we used 8 at the time
2. what is the engine and version underneath your sql db?this was not for a dba role, just standard backend engineer
3. why did you use python instead of r for x project? this was about a gui automation script
>You can often tell an inexperienced interviewer from the extremely domain specific question they ask which _they_ are familiar with.
Lol a bit touchy aren't we?
Like I said, it's not really relevant in today's languages. It was for a Delphi/Pascal position. If you do any type of database code (like T-SQL), you would also know it. If your experience is mainly in C type languages, everything is a function so it doesn't apply.
If you hired a guy for a Delphi position who didn't know the difference between a function and a procedure, you hired the wrong guy.
procedure Hello;
begin
ShowMessage ('Hello world!');
end;
function Double (Value: Integer) : Integer;
begin
Double := Value * 2;
end;
Function or procedure is defined in every subroutine. It's a very basic question for Delphi, like what's the difference between an integer and a string.
> Our "gotcha," which doesn't apply to most languages anymore is, "What's the difference between a function and a procedure." It's a one sentence answer, but people who didn't know it would give some pretty enlightening answers.
If you're asking this question (by virtue of the present-tense "is") in the year 2025 even though by your own admission
> it's not really relevant in today's languages.
then you aren't giving candidates a good impression. Even though I would have nailed this question I would have serious reservations about any job that would ask it in an interview because it means that the person interviewing me has more concern for legacy minutiae than broad technical knowledge or problem-solving skills.
No. I just looked up other responses to your post. It's obvious you got exposed as being inexperienced (or an idiot), while posing to know the definitive with your "gotcha". Being inexperienced (or ignorant) is not a problem, but being cocky is.
I think you're projecting. You aren't posting this using your real name are you?!
Here's something to consider, Dennis. Instead of using any type of reasoning that maybe I'm interviewing for a language you aren't familiar with where functions and procedure differences matter, you decided to just go off the handle and call me inexperienced and/or an idiot. This is what we call in the hiring business a "huge red flag." I recommend maybe use some of that big brain you have and apply some deductive reasoning instead of just calling people names.
Look up my other responses - I decided to call you names _later_ . Other people also pointed that out to you. I'm sure you would thick twice before asking your 'gotcha' again (the question is fine in general but not as a gotcha).
>>You can often tell an inexperienced interviewer from the extremely domain specific question they ask which _they_ are familiar with.
>Lol a bit touchy aren't we?
You lost your composure and decided to start calling names after this. I haven't asked it since the late 90s, early 2000s. It was a for a Delphi position. It's a bonehead easy question any Delphi developer who got to chapter 2 of any Delphi book would have understood. It's still an applicable question for a SQL developer and it's just as easy. I even showed you sample code. I don't see why you aren't getting it.
I'm not getting it, I also see that 2 other people are also not getting it.
>I haven't asked it since the late 90s, early 2000s.
You overall gave the impression that you are currently asking it.
And a personal rhetorical question - aren't you too old to even state this 'gotcha' business _today_ about what you did in the past? What made you state it? If I gave you the benefit of doubt - that was slip, where you omitted the past tense.
(If I did that in the 90's I'd be a embarrassed to even mention it today.)
Words like functions/procedures tend to have different connotations across languages and once one crosses one's 15th language, and each having some 20 different keywords, it become difficult to remember what the exact connotation of a word is, in a specific language/framework. This is the most likely situation of the guy whose post I responded to.
The exception to the rule is, if you have been working quite a bit _recently_ on a specific language. You are presumably talking about this situation.
It’s ok to say that it’s never professionally mattered. No one has ever been paid to know that. “Are side effects a bad pattern?” Lotsa people have needed to know that on day one.
It's a question to filter out people who don't know a lick of Delphi for a position where the person was coding Delphi. It's like level 2 after the hello world chapter. It's easier than asking a database developer the difference between a query and a view. It's a bonehead simple question that 70% of applicants couldn't answer.
You would be surprised how many bad applicants interviewers get. It's only gotten worse over the last 20 years.
> Single interview, we ask the candidate about *his* CV and what *his* expectations are, what are *his* competences and we ask *him* to show us some code *he* has written
You... might want to think about what implicit biases you might be bringing here
What I've been thinking about leetcode medium/hard as a 30-45 minute tech interview (as there are a few minutes of pleasantry and 10 minutes reserved for questions), is that you are only really likely to reveal 2 camps of people—taking in good faith that they are not "cheating". One who is approaching the problem from first principles and the other who knows the solution already.
Take maximum subarray problem, which can be optimally solved with Kadane's algorithm. If you don't know that, you are looking at the problem as Professor Kadane once did. I can't say for sure, but I suspect it took him longer than 30-45 minutes to come up with his solution, and I also imagine he didn't spend the whole time blabbering about his thought process.
I often see comments like: this person had this huge storied resume but couldn't code their way out of a paper bag. Now having been that engineer stuck in a paper bag a few times, I think this is a very narrow way to view others.
I don't know the optimal way to interview engineers. I do know the style of interview that I prefer and excel at[0], but I wouldn't be so naive to think that the style that works for me would work for all. Often I chuckle about an anecdote from the fabled I.P. Sharp: Ian Sharp would set a light meter on his desk and measure how wide an interviewees eyes would get when he explained to them about APL. A strange way to interview, but is it any less strange than interviewing people via leetcode problems?
0: I think my ideal tech screen interview question is one that 1) has test cases 2) the test cases gradually ramp up in complexity 3) the complexity isn't revealed all at once; the interviewer "hides their cards," so to speak 4) is focused on a data structure rather than an algorithm such that the algorithm falls out naturally rather than serves as the focus. 5) Gives the opportunity for the candidate to weigh tradeoffs, make compromises, and cut corners given the time frame. 6) Doesn't combine big ideas (i.e. you shouldn't have to parse complex input and do something complicated with it); pick a single focus. Interviews I have participated and enjoyed like this: construct a Set class (union, difference, etc); implement an rpn calculator (ramp up the complexity by introducing multiple arities); create a range function that works like the python range function (for junior engineers, this one involves a function with different behavior based on arity).
>Take maximum subarray problem, which can be optimally solved with Kadane's algorithm. If you don't know that, you are looking at the problem as Professor Kadane once did. I can't say for sure, but I suspect it took him longer than 30-45 minutes to come up with his solution, and I also imagine he didn't spend the whole time blabbering about his thought process.
This is something that drives me nuts in academia when it comes to exam questions. I once took an exam that asked us to invent vector clocks from whole cloth, basically, having only knowledge of a basic Lamport clock for context. I think one person got it--and that person had just learned about vector clocks in a different class. Given some time, it's possible I could have figured it out. But on an exam, you've got like 10-15 minutes per question.
The funny thing about it is that I do the same damn thing from the other side all the time when working with students. It's incredibly tempting once you know the solution to a problem (especially if you didn't "solve" it yourself, but had the solution presented to you already) to present the question as though it has an obvious solution and expect somebody else to immediately solve it.
I'm aware of the effect, I've experienced it many times, and I still catch myself doing it. I've never interviewed a candidate for a job, but I can only imagine how tempting it would be to fall into that trap.
When I'm interviewing a candidate, I'm often asking myself if this question is just something I happen to know therefor expect the candidate to know too, or if it's crucial to doing the job?
Sometimes it may not be fair to expect a random developer to be familiar with a specific concept. But at the same time it might be critical to the kind of work we're doing.
> There should be a ton of companies out there just dying to hire someone with that kind of experience.
heh.. they are probably dead already?
i have even longer years.. But this time i am looking since.. september? Applying 1-2 per day, on average.. Widening the fishing net each month.. ~2% showed some interest.. but no bingo.
I am with you! Been programming since I was 10 and have 20YoE. Many of my prototypes have grown into full fledged products, I have 40+ published papers, and I am regularly sought out for advice and help by those who know me. Everyone i have been, I am always told I am a good catch.
However, I won't do leet coding. I want to hear about why I should come work for u. What about my works makes u think I could help ubm with your problem. Then let's have a talk about your problems and where I can create value for you.
My experience in hiring is that leet coders are good one trick ponies. But long term don't become technical peers.
Part of the problem is there just aren't a lot of people out there who can correctly judge that level of experience, and looking up the spectrum tends to simply look weird.
I agree. Do you have any thoughts on how to mitigate this? After all, its in your best advantage and the companies to hire someone with talents because of the value they can bring.
The problem isn't AI, the problem is companies don't know how to properly select between candidates, and they don't apply even the basics of Psychometrics. Do they do item analysis of their custom coding tests? Do they analyse the new hires' performances and relate them to their interview scores? I seriously doubt it.
Also, the best (albeit the most expensive) selection process is simply letting the new person to do the actual work for a few weeks.
> Also, the best (albeit the most expensive) selection process is simply letting the new person to do the actual work for a few weeks.
What kind of desperate candidate would agree to that? Also, what do you expect to see from the person in a few weeks? Usual onboarding (company + project) will take like 2-3 months before a person is efficient.
Candidate would be compensated, obviously. That's why it's expensive.
You don't need him to become efficient. Also I don't think it is always necessary to have such long onboarding. I'll never understand why a new hire (at least in senior position) can't start contributing after a week.
> Candidate would be compensated, obviously. That's why it's expensive
Ok... take me through it. I apply to your company and after a short call you offer me to spend 4 weeks working at your place instead of an interview.
I go back to my employer, give them resignation letter, work the rest of my notice period (2 months - 3 months), working on all handovers, saying goodbyes.
Unless the idea is to compensate me for the risk (I guess at least 6 months salary, probably more), then I do not see how you'd get anyone who is just a poor candidate to sign up for this.
> You don't need him to become efficient
So what will you see? Efficiency, being independent and being a good team player are the main things that are difficult to test during a regular interview.
And so that self-selects for people who already are unemployed then, right? Most developers I know (including myself) look for a new job while still having a job, as to not create a financial hole in-between. I'd be curious if that doesn't then end up with lower quality candidates who ended up unemployed to begin with?
And, additionally, it encourages your candidates to still be interviewing while they're on their probationary period with you, since they may be back to unemployed after 4 weeks or whatever. Which creates even more potential issues if they get a much better offer while they're onboarding with you.
> self-selects for people who already are unemployed then
You can say that about all forms of hiring process. If you're unemployed, you obviously have more time: to spend more time on the take-home assignments (which I hate, see another thread [1]), to add more stuff to your GitHub profile, to go to more interviews, etc.
> You can say that about all forms of hiring process
Yes, but there's a significant difference between spending a few hours on a take-home assignment and dropping your current employment to spend 4 weeks potentially in another city working full time.
I'd argue the bigger expense is on the team having to onboard what could potentially be a revolving door of temporary hires. Getting a new engineer to the point where they understand how things work and the specific weirdness of the company and its patterns is a pretty big effort at anywhere I've worked.
If you work with Boring Technology, your onboarding process has no reason to be longer than a week, unless you're trying to make the non-tech parts of the role too interesting.
> unless you're trying to make the non-tech parts of the role too interesting.
Unless your role is trivial to replace with an LLM, you need to understand the business. Maybe not for really junior role, but everything above - you need to solve issues. Tech is just a tool.
I am not sure I follow - when you hire you search for someone who has 100% coverage of the tech you're using and also already works for your direct competitor?
Let's say you're hiring manager for a company that compares flight tickets, something similar to Google Flights or Skyscanner. You need three additional Rust engineers. You're located in Palermo, Italy.
How do you hire people that would not only know Rust, be willing to move to Palermo, or at least visit occasionally, but also know the airfare business?
Even if you're willing to have people remotely, in the same region, how many unemployed Rust developers that know that business are on the market? 0?
> when you hire you search for someone who has 100% coverage of the tech you're using and also already works for your direct competitor?
Ideally, yes. It's a common occurrence among large organisations. Google and Apple used to even have an anti-poaching agreement.
> How do you hire people that would not only know Rust, be willing to move to Palermo, or at least visit occasionally, but also know the airfare business?
Rust isn't Boring, which is why you don't do that and hire one of many Java developers and do Java, unless the tradeoff is really worth it.
How do you control for confounders and small data?
For data size, if you're a medium-ish company, you may only hire a few engineers a year (1000 person company, 5% SWE staff, 20% turnover annually = 10 new engineers hired per year), so the numbers will be small and a correlation will be potentially weak/noisy.
For confounders, a bad manager or atypical context may cause a great engineer to 'perform' poorly and leave early. Human factors are big.
Sure, psychological research is hard because of this, but that's not what I'm proposing - I'm talking about just having some data on predictive validity of the hiring process. If there's some coding test: is it reliable and valid? Aren't some items redundant because they're too easy or too hard? Which items have the best discrimination parameter? How the total scores correlate with e.g. length of the test takers tenures?
Sure, the confidence intervals will be wide, but it doesn't matter, even noisy data are better than no data.
Maybe some companies already do this, but I didn't see it (though my sample is small).
I mostly skipped the technical questions in the last few interviews I have conducted. I have a conversation, ask them about their career, about job changes, about hobbies, what they do after work. If you know the subject, skilled people talk a certain way, whether it is IT, construction, sailing.
I do rely on HR having, hopefully, done their job and validated the work history.
I do have one technical question that started out as fun and quirky but has actually shown more value than expected. I call it the desert island cli.
What are your 5 linux cli desert island commands?
Having a hardware background, today, mine are: vi, lsof, netcat, glances, and I am blanking on a fifth. We have been doing a lot of terraform lately
I have had several interesting responses
Manager level candidate with 15+ years hands on experience. He thought it was a dumb question because it would never happen. He became the teams manager a few months after hiring. He was a great manager and we are friends.
Manager level to replace the dumb question manager. His were all Mac terminal eye candy. He did not get the job.
Senior level SRE hire with a software background. He only needed two emacs and a compiler, he could write anything else he needed.
> I have a conversation, ask them about their career, about job changes, about hobbies, what they do after work. If you know the subject, skilled people talk a certain way, whether it is IT, construction, sailing.
My experience differs a lot. Many insanely skilled people are somewhat "weird" (including possibly
- being a little on the spectrum,
- "living a little bit in their own world",
- having opinions on topics that are politically "inappropriate" (not in the sense of "being on the 'wrong' side of a political fence", but rather in the sense of "an opinion that is quite different than what you have ever heard in your own bubble", and is thus not "socially accepted")
- being a little bit "obnoxious" (not in bad sense, but in a sense that might annoy a particular kind of people))
What you consider to be "skilled people" is what I would rather call "skilled self-promoters" (or possibly "smooth talker"). "Skilled people" and "skilled self-promoter" are quite different breeds of people.
> My experience differs a lot. Many insanely skilled people are somewhat "weird" (including possibly
I am actually a bit weird myself, so I can relate.
> What you consider to be "skilled people" is what I would rather call "skilled self-promoters". "Skilled people" and "skilled self-promoter" are quite different breeds of people.
I don't mean that they have told me that they are skilled, or that their resume has implied it. I mean that they actually have the skills. Self-promoters that don't know the information always look good on paper, but after a few minutes of talking to them you can tell that they don't quite match.
Before IT, I was a live sound engineer TV, theater, music. There was also a entertainment university starting up around the same time. They were pumping out tons of "trained" engineers that looked good on paper but couldn't mix for shit. I think we can blame them for the shitification of pop music.
> Self-promoters that don't know the information always look good on paper, but after a few minutes of talking to them you can tell that they don't quite match.
My experience differs here: these are not "good self-promoters", but impostors.
Good self-promoters typically have some above-average (though commonly not really exceptional) skills in their area, but their expertise is in the capability of smooth talking (including smalltalk), promoting their contributions, and talking at eye level with various stakeholders.
If you are really exceptional in your area, you will often (though not always) consider smalltalk to be waste of your time, and will often have difficulties talking at eye level with various stakeholders, because either they are not sufficiently knowledgable in your area of expertise to understand you, or the other way round (for the latter point: becoming really great in one area often means that you won't have the time to get sufficiently deep into a lot of other areas, even though for some of them you might become quite skilled if you had more time).
I have an ice breaker type question which is “what’s something (tool, tech, whatever) you are interested/excited about and wish people knew more about?” Selfishly, interviewing is kind of boring, so I’m always looking to learn something new.
Sadly, out of 100s of people, I’ve probably only gotten an interesting response a handful of times. Mostly people say some well known tech from the job description.
I never held that against anyone, but the people who had an interest in something were more fun to work with.
>I mostly skipped the technical questions in the last few interviews I have conducted. I have a conversation
Sir, you have attained dizzying intellectual heights that few men have.
My comment is meant to be a compliment, not snarky. And indeed I have noticed that the best people I have encountered can often size people up accurately with very general questions often on unrelated subjects.
We hired the guy who said it was a dump question, he became our manager. He then decided to retire and we had to replace him. One of the candidates answered the 5 cli question with terminal eye candy, not functional commands. He was not hired for the job.
My last interview, for the job I'm currently employed in, asked for a take home assignment where I was allowed to use any tool I'd use regularly including AI. Similar process for a live coding interview iterating on the take home that followed. I personally used it to speed up wirting initial boilerplate and test suites.
I fail to see why this wouldn't be the obvious choice. Do we disallow linters or static analysis on interviews? This is a tool and checking for skill and good practices using it makes all sense.
As someone on the other side of the table, I don't care if you used AI to complete a take-home project. I care if you can explain the strengths and weaknesses of the approach it took, or if you chose to steer it in one direction or another. It usually becomes quite clear those who actually understand what the AI actually did for them.
There’s no other industry* that interviews experienced people the way we do. So maybe just do what everyone else does.
Everyone is so terrified of hiring someone that can’t code, but the most likely bad hires and the most damaging bad hires are bad because of things that have nothing to do with raw coding ability.
*Except the performing arts. The way we interview is pretty close to the way musicians are interviewed, but that’s also really similar to their actual job.
I've been arguing that "AI" has very little impact on meaningful technical interviews, that is ones that don't test for memorization of programming trivia: https://blog.sulami.xyz/posts/llm-interviews/
We have been interviewing people who are obviously using covert AI helper tools. Ask them a question and they respond with coherent response, but they are just reading off of a window we can't see.
In some cases it is obvious they are blathering a stream of words they don't understand. But others are able to hold something resembling a coherent conversation. We also have to allow for the fact that most people we interview aren't native English speakers, and are talking over Teams. It can be very hard to tell if they are cheating.
Asking questions to probe their technical skills is essential, otherwise you are just selecting for people who are good at talking and self promotion. We aren't just asking trivia questions.
We also give a simple code challenge, nothing difficult. If they have a working knowledge of the language, they should be able to work through the problem in 30 minutes, and we let them use an IDE and google for things like regex syntax.
Some of them are obviously using an AI, since they just start typing in a working solution. But in theory they could be a Scala expert who remembers how to use map plus a simple regex...
A couple of weeks ago I interviewed at a place where I had to do a take-home exercise. It's fine, I don't mind. No Leetcode. Just my own IDE, my own shortcuts, and write a piece code that solves a problem.
I was asked whether I used AI/LLM for the solution. I didn't. I felt like using an LLM to solve the problem for me wasn't the right way of showcasing knowledge. The role was for some form of 'come in with knowledge and help us'.
The response to that was basically: everybody here uses AI.
I declined the follow-up interview, as I felt that if all you have is the speed of AI to be ahead of your competitors, you're not really building the kind of things that I want to be a part of. It basically implies that the role is up in the air as soon as the AI gets better.
When I started coding I did it in notepad. I thought it was hardcore and cool. I was young and stupid. Then I adopted an IDE and I became much better at writing code.
To me AI is just another tool that helps me solve problems with code. An auto complete on steroids. A context aware stack overflow search. Not wanting to adopt or not even work somewhere where colleagues use it, sounds to me like coding in notepad AND in the process scoffing those who use an IDE.
Besides, if AI gets to the point it can replace you, it will replace you. Better to start learning how to work with it so you can fill whatever gap AI can't.
I still mainly use a text editor after several decades, and do a lot of thinking and initial design with pencil and paper. IDEs just get in the way.
I've seen the type of code AI generates. It might work, but if you think that's good or that massaging it so it works will make you any better, I have some bad news for you...
Prediction: faangs will come up with something clever or random or just fly everyone onsite, they are so rich and popular, they can filter by any arbitrary criteria.
Second-rate companies will keep some superficial coding, but will start to emphasize more of the verbal parts like system design and retrospective. Which sucks, because those are totally subjective and mostly filters for whoever can BS better on the spot and/or cater to the interviewer's mood and biases better.
My favorite still: in-person pair programming for a realistic problem (could be made-up or shortened, but similar to the real ones on the job). Use whatever tools you want, but get the correct requirements and then explain what you just did, and why.
A shorter/easier task is to code review/critique a chunk of code, could even just print it out if in person.
I've taken this approach, and found that it's trivially easy to distinguish people relying on LLMs from people who have thought the problem through and can explain their own decision-making process.
I had a couple of people who, when asked to explain specific approaches reflected in their code, very obviously typed my question right back into ChatGPT and then recited its output verbatim. Those interviews came to an end rather quickly.
One of my favorite ones was when I asked a candidate to estimate the complexity of their solution, and ChatGPT got it wrong, giving O(log(n)) for an O(n) algorithm. When I asked leading questions to see if the candidate could see where the error came in, they starting verbatim reciting a dictionary definition of computational complexity, and could not address the specifics of the problem at all.
This whole conversation is depressing me. when I left work a couple years ago due to health reasons, AI was just beginning to become a problem. Now, thanks to a clinical study, I may possibly be able to return to work, and it sounds like the industry has changed drastically.
I think that the effects at the moment are highly exaggerated in the tech media than the reality on the ground.
How long that will remain true is a very open question, where different folks have widely differing timelines on when they expect AI to have highly meaningful impacts.
Our tech screen is having the candidate walk me through a small project that I created to highlight our stack. They watch my screen and solve a problem to get the app running then they walk me through a registration flow from the client to the server and then returning to the client. There are no gotchas but there are opportunities to improve on the code (left unstated... some candidates will see something that is suboptimal and ask why or suggest some changes).
We get to touch on client and browser issues, graphQL, Postgres, Node, Typescript (and even the various libraries used). It includes basic CRUD functionality, a third party API integration, basic security concerns and more. It's just meant to gauge a minimal level of fluency for people that will be in hands on keyboard roles (juniors up to leads, basically). I don't think anyone has found a way to use AI to help them (yet) but if this is too much for them they will quickly fall flat in the day to day job.
Where we HAVE encountered AI is in the question/answer portion of the process. So far many of those have been painfully obvious but I'm sure others were cagier about it. The one incident that we have had that kind of shook us was when someone used a stand-in to do the screen (he was fantastic, lol) and then when we hired him it took us about a week to realize that this was a team of people using an AI avatar that looked very much like the person we interviewed. They claimed to be in California but were actually in India and were streaming video from a Windows machine to the Mac we had supplied for Teams meetings. In one meeting (as red flags were accumulating) their Windows machine crashed and the outline of the person in Teams was replaced by the old school blue screen of death.
How about paid internships as a way to filter candidates? As in, hire a candidate for a small period of time, like 2 weeks or something, and have them work on a real task with a full-time employee and use their performance on that to decide whether or not to hire.
I realize it's not easy for smaller companies to do, but I think it's the single best way to see if someone's fit for a job
I'm someone who hated leetcode style interviews for the longest, but I'm starting to come around on them. I get that these style of questions are easy to game, but I still think they have _some_ value. The point of these style of questions was supposed to test your ability to problem solve and come up with a good solution given the tools you knew. That being said, I don't think every company should be using this type of question for their interviews. I think leetcode style questions should be reserved for companies that are pushing the boundary of the industry since they're exploring charted territory and need people who can come up with unique solutions to problems no one really knows. I think most companies would be fine with some kind of pairing problem since most people are probably solving engineering problems instead of computer science problems. But none of this matters, since, we all know that even if we went that direction as an industry, the business people would fuck it up some how anyways.
> reserved for companies that are pushing the boundary of the industry
In a world where every company beleives (or wants to beleive) that they are doing some ground-breaking, bleeding edge work (see any tech company blog and you can only find hyped technologies in there), I do not think one can expect companies to do a fair assessment of if they really are doing such work.
I had no idea people took hackerrank as a serious signal rather than as a tool for recent graduates to track interview prep progress. Surely it has all the same issues AI does: you have no way of verifying that the person who takes the interview actually is responsible for that signal.
I don't see AI as a serious threat to the interview process unless your interview process looks a lot like hackerrank.
Your “unless” covers a huge swath of this industry, at the low end and at the high end. Excluding places that do that leaves you with what exactly? Boutique shops filled with 20 year veterans?
What do you mean by the "high" end? I would consider this sort of interview style necessarily precluding such a place from being considered a high-quality work-place. Not only is it a miserable way to interview, it's not an effective signal for engineer quality beyond rapid code snippet production.
> Excluding places that do that leaves you with what exactly? Boutique shops filled with 20 year veterans?
We are on a VC forum—I imagine small shops focused on quality are quite common here.
“High end” was meant as shorthand for FAANG …high comp, not necessarily high tech complexity.
You are correct about the deficiencies of the whiteboard interview. It is not a sane way to hire an individual. It makes sense as a way to hire someone in the top 20% from a large unfiltered pool. So wrt high/low, that’s what FAANG companies have to do, and for many nontechnical companies they outsource this work or emulate FAANG practices for no good reason.
My point was that there are very few places that don’t do this.
I just abandoned the code interview altogether and ask them questions about process. It's a very simple workaround, but very effective. I'll admit it helps that there are very few problems these days outside of specific problems to solve that require a high degree of technical competency to tackle.
I feel like we, SWEs, have been over-engineering our interview process. Maybe it's time to simplify it, for example, just ask questions based on the candidate's resume instead of coming up with random challenges. I feel like all the new proposals seem overly complicated, and nobody, interviewer or interviewee, is happy with any of them.
Definitely over-engineering. But I also think the industry is just extremely bad at hiring anything above junior or entry-level. Job postings are so generic and interchangeable between companies that they don't actually tell you what the role is or what the company is looking for. Everyone wants to cast the widest possible net so that they catch some wunderkind genius out of thousands. Then, they wonder why they can't find the exact person they're looking for to solve the problem they're filling the role for.
In reality, job postings should be incredibly specific, with specificity rising as the role requires more experience and problem solving. You'll get less applicants (or will be able to clearly screen out the people who don't meet the specific requirements) but you'll get ones that actually match what you are looking for and can actually solve the problem your company is trying to solve with filling the role. Then the conversation/interview is much more important and both sides feel like they have some "stakes in the game".
This risks hiring candidates who can present themselves and their past projects very well but fail to actually write code and ship anything on the job. I've seen it happen.
You would be very amazed at how many people with reasonably strong resumes can't write _any_ code. Google for fizzbuz, it's a dumb problem, but candidates often can't solve similar problems with a _take_home_ interview.
Licensing. We do the Leetcode interview in a controlled testing center. When you apply for a position, I look up your license number, then I know you can leetcode without wasting any of my developer resources on determining that.
> Licensing. We do the Leetcode interview in a controlled testing center.
Congratulations, you are now a "Certified Leetcoder (tm)". :-(
Seriously: what a lot of people write down here is that a lot of programming jobs don't involve code puzzle skills, but are often rather about putting stuff/APIs together in the currently fashionable framework.
This makes becoming a Certified Leetcoder (tm) just another useless hoop to jump through.
(Just to be clear: for those few programming jobs that demand the employee to solve algorithmic puzzles regularly, doing them in a job interview makes sense. But these jobs are rare.)
> This makes becoming a Certified Leetcoder (tm) just another useless hoop to jump through.
And this differs from the status quo how? Employers obviously find value in this signal for better or worse. We’re just making it so it only needs to be done once, by trained proctors, instead of for every position you apply for.
Former head of product there here: no, we didn't do this kind of identity verification. That would have prohibitively damaged people's willingness to actually do the interview. We did use various means to try to identify people signing up multiple times, and we caught plenty of people trying to duplicate themselves that way, but you didn't have to physically go to a center to do our interview.
I've considered something like that for my current company, which is doing basically the same thing, but:
(a) this has not, in practice, been a problem for us in identifying good candidates
and, far less importantly:
(b) you need very high scale to have interviewers everywhere that candidates are OR you're paying extra for a third-party controlled environment
(c) scheduling and cancellations become more difficult and costly respectively
When it comes to interviews I generally stick to asking fairly easy questions but leaving out key details, I care a lot more about candidates asking following ups and talking through the code they are writing over what code they actually produce. If a candidate can ask questions when they don't understand something and talk through their thought process then they are probably going to be a good person to work with. High level design questions are often pretty valuable I find as well, which I usually don't require code for I just ask them to talk through their ideas of how they would design an application.
In my uni days, I respected professors who designed exam in a way where students can utilize whatever they could to complete the assignment, including internet, their notes, calculators, etc.
I think the same applies to good tech interview. Company should adapt hiring process to friend with AI, not fight.
Ask questions that are tricky to cheat, evaluate candidates by their thought process in solving a problem and ability to clarify, discuss and justify their decisions.
nah ai killed stupid tech interviews. you can easily get an idea of someones competence by literally just talking to them instead of making them do silly homework exercises and testing their rote memorisation abilities.
This is the real answer. However to gauge competence you must first have it. The fact that most people don't is why we are in this position in the first place.
I don't think it did, if anyone cares.
The way I've been advocating to my colleagues who are concerned about "cheating" is that there's probably a problem with the interview process.
I prefer to focus on the think, rather than the solve.
Collaborate, as opposed to just do.
Things that really tell me if I can work with that person and if together, we can make good things.
This oft repeated talking point lacks perspective on what companies want and how interviews work. It also doesn’t address the AI problem.
Interviews are screening for multiple things, not just the ability to do one specific technical job. More often than not, technical coding ability is not even at the top of the priority list. Interviews are looking for well-rounded candidates who can do more than 1 job. Companies want to know if you can change jobs easily, they want to know if you’re average, better than the average programmer, or exceptional. They want to know if you’ll make a good manager after a few years, how good you are with people, how well you prioritize and communicate.
I had a professor in college that graded tests with the median skewed low, centered on a D. He complained that the usual practice of putting it on C or B made it so he could clearly see the difference between F-, F, and D- students, while the A students were all clumped together. He wanted to identify the hard workers and superstars in the class, see who was A vs A+ vs A++. It freaked everyone out when grades came out much lower than expected, but he renormalized at the end and people with test scores in the Cs and Ds got As and Bs in the class.
Be careful what you wish for. It’s competitive right now and interviews that limit screening to ability to do basic job-level coding and don’t screen for knowledge and soft skills and exceptionalism will make it harder for people who are good to demonstrate they’re better than people who are mediocre or use AI. Is that what you want?
Unless the job you're interviewing for is remote-only, this makes perfect sense. If you expect your candidates to be able to work in your office, they should be interviewed there.
So, when AI can pass the tech interview seamlessly, I guess we can just hire it?
Maybe the future will be human shills pretending to be job candidates for shady AI “employment agencies” that are actually just (literally) skinning gpt6 apis that sockpuppet minimum wage developing nation”hosts”?
I think that a mythology about where the difficulty in working with computers lies has made the relationship between businesses and the people they hire to do this stuff miserable for quite some time
"Coding", as in writing stuff in programming languages with correct syntax that does the thing asked for in isolation, has always been a very dumb skill to test for. Even before we had stackoverflow syntactic issues were something you could get through by consulting a reference book or doing some trial and error with a repl or a compiler. That this is faster now with internet search and LLMs is good for everyone involved, but the fact that it's not what matters remains
The important part of every job that gets a computer to do a thing is a combination of two capabilities: Problem-solving, that is, understanding the intended outcome and having intuition about how to get there through whatever tools are available, and frustration tolerance: The ability to keep trying new* stuff until you get there
Businesses can then optimize for things like efficiency or working well with others once those constraints are met, but without those capabilities you simply can't do the job, so they're paramount. The problem with most dinky little coding interviews wasn't that you could "cheat", it's thst they basically never tested for those constraints by design, though some clever hiring people manage to tweak them to do so on an ad hoc basis sometimes
* important because a common frustration failure mode is repetitive behavior. Try something. Don't understand why it doesn't work. Get more frustrated. Try the same thing again. Repeat
thank fuck. they are terrible. being interviewed by CTOs just out of university, with no experience for a senior in everything role. they ask you to do some lame assignment, a pet problem not once looking at 20 years of GitHub repos and open source contributions.
Funny enough, the songs from the website Coding For Nothing about grinding LeetCode and endless take-home projects seem very relevant, and everything nowadays feels like a meme.
Tech interviewing has become a weird survival game, and now AI is flipping the rules again. If you need a laugh: https://codingfornothing.com
One option is to make the interviews harder and let candidates use ai to see how they can work with ai and actually build working product. They will be using ai in the job anyway so let them use it instead of asking stupid algorithm questions to sort an array
So there’s AI that’s really good at doing the skills we’re hiring for. We want you to not use AI so we can hire you for a job that we’re saying we’re going to replace with AI. Sounds like a great plan.
No. Using AI requires a depth of knowledge to spot the mistakes in the generated. code, and to know how to fit all the snippets of code in to something that works.
We need to know that the developer actually has skills and isn't just secretly copying the answer off of a hidden screen. We are interviewing now, and some cantidates are obviously cheating. Our interview process is not leet code based, and reasonably chill, but we will probably have to completely rethink the process.
Since we are hiring contractors, in theory we can let them go after a couple months if they suck, but we haven't tested out how this will work in practice.
Maybe we don't need employers. Maybe we need a bunch of 1-person companies. I don't think AI is yet the force multiplier that makes that feasible for the masses, but who knows what things will look like in a few years.
Probably the same as it ever was. I couldn't tell you what that is, but it never had anything to do with economics anyway.
Having a job has always been a necessary evil. If the time is coming when that changes, all the better. If we can't figure out how to behave in the presence of plenty, we don't deserve utopia anyway.
I think code design can often cover just as much as actual code anyway. Just describe to me how your solve it, the interfaces you'd use, and how you'd show me you solved it.
As an interviewee it's insane to me how many jobs I have not gotten because of some arbitrary coding problem. I can confidently say after having worked in this field for over a decade and at a FAANG that I am a very capable programmer. I am considered one of the best on every team I've been on. So they are definitely selecting the wrong people IMO.
* Take a candidate's track record into account. Talk with them about it.
* Show that you're experienced yourself, by being able to tell something about what someone would be like to work with, by talking with them.
* Get a reputation for your company not tolerating dishonesty. If someone cheats in an interview and gets caught, they're banned there, all the interviewers will know, and the cheater might also start to get a reputation beyond that company. (Bonus: Company reputation for valuing honesty is attractive to people who don't want dishonest coworkers.)
* Treat people like a colleague, trying to assess whether it's a good match. You're not going to be perfectly aligned (e.g., the candidate or the company/role might be a bit out of the other's league right now), but to some degree you both want it to be a good match for both parties. Work as far as you can with that.
(Don't do this: Leetcode hazing, to establish the dynamic of them being there to dance for your approval, so hopefully they'll be negged, and will seek your approval, won't think critically about how competent and viable your self/team/company are, and will also be less likely to get uppity when you make a lowball offer. Which incidentally places the burden of rehearsing for Leetcode ritual performances upon the entire field, at huge cost.)
The death of shitty interviews has been greatly exaggerated.
AI might make e.g. your leetcode interview less predictive than it previously would have been. But was it predictive in the first place? I don't think most interviews are written by people thinking in those terms at all. If your method of interviewing never depended on data suggesting it actually, you know, worked in the first place, why would it matter if it starts working even worse?
Insofar as it makes the shittiness of those interviews more visible, the effect of AI is a good thing. An interview focused on recall of some specific algorithm was never predictive, it's just now predictive in a way that Generic Business Idiots can understand.
We frequently interview people who both (a) claim to have been in senior IC roles (not architect positions, roles where they are theoretically coding a lot) for many, many years and (b) cannot code their way out of a paper bag when presented with a problem that requires even a modicum of original reasoning. Some of that might be interview nerves, of course, but a lot of these people are not at all unconfident. They just...suck. And I wonder if what we're seeing is the downstream effects of Generic Business Idiots hiring primarily people who memorize stuff than people who build stuff.
Another possibility is that their job subtly drifted.
I wrote a lot of code as a grad student but my first interviews afterward were disasters. Why? Because I’d spent the last few months writing my thesis and the few months before that writing a very specific kinds of code (signal processing, visualization) that were miles away from generic interview questions like “Make the longest palindrome.”
We don't ask "make the longest palindrome". We ask "convert this English into code that does what it says". If you want to make the discussion more concrete, we have a public practice problem [1] that we send out with our interview bookings so that people know what to expect. The real problems we ask are very similar to it.
Do you feel like there's anything there that any reasonably skilled programmer shouldn't be able to figure out on the fly?
That's not a typical leetcode problem though. Most companies ask things like "solve this slightly modified knapsack problem" which takes 5 minutes if you know the solution and 50 minutes if you don't.
Yeah, it's very intentionally not a typical leetcode problem, precisely because leetcode is nearly worthless for screening people these days. I was replying to someone who seemed to be objecting to my opinion of some candidates' skills on the basis that maybe they just didn't know the specific thing we ask.
We did an experiment at interviewing.io a few months ago where we asked interviewees to try to cheat with AI, unbeknownst to their interviewers.
In parallel, we asked interviewers to use one of 3 question types: verbatim LeetCode questions, slightly modified LeetCode questions, and completely custom questions.
- Interviewers couldn't tell when candidates were cheating at all
- Both verbatim and slightly modified LeetCode questions were really easy to game with AI
- Custom questions were not gamable, on the other hand[1]
So, at least for now, my advice is that companies put more effort into coming up with questions that are unique to them. It's better for candidates because they get better signal about the work, it reduces the value asymmetry (companies have to put effort into their process instead of just grabbing questions from LeetCode etc), and it's better for employers (higher signal from the interview).
[1] This may change with the advent of better models
A couple years ago, we were using a take home coding assignment as a hiring signal. It was a small API based off something I'd built for an internal tool. It was self-contained and relatively easy to explain. The .md file was about two pages.
I recently fed it into ChatGPT and asked it to do the assignment. It did it perfectly -- I read the code in detail and couldn't find any issues.
So custom questions are off the table now, too. We'll be using a code review instead for the next round.
The inconvenient truth is that everything circles back to in-person interviews.
The article addresses this:
>A lot of companies are doing RTO, but even companies that are 100% in-office still interview candidates from other cities. Spending money to fly every candidate out without an aggressive pre-screen is too wasteful.
No, accidently hiring someone who AI'd their way through the interview costs orders of magnitude more to undo. It's absolutely worth paying for a round trip flight and a couple days of accommodations.
1point3acres is massacring tech interviews right now. Having to pay $80/month to some China based website where NDA-protected interview questions are posted regularly, then being asked the same questions in the interview, seems insane.
It also feels like interviewers know this and assume you studied the questions, they seem incapable of giving hints, etc if you don't have the questions memorized.
Very funny :) I too failed an interview at google, also related to binary search on a white board. I never write with pens. I'm on keyboards the whole time, my handwriting is terrible.
I've built a search engine for two countries and then I was failed by a guy that wears cowboy hats to work at google in Ireland. Not a lot of cows there I'm guessing. (No offence to any real cowboys that work at google of course).
I did like the free flight to Ireland though and the nice lunch. Though I was disappointed I lost "Do no evil" company booklet.
The best interview process I've ever had was going to work with former coworkers, aka no real process. A couple of quick calls with new people who deferred strongly to the person who knew me, my work, and my values. Nothing else has the signal value.
Of course the problem is this can't scale or be outsourced to HR, but is this a bug or a feature?
The best interview processes are chill, laid back, open ended.
That's the only way you're going to get relevant information.
I've been verified to the moon and back by Apple and others for roles that could never have worked.
The problem is that when it comes to the hiring process, everyone is suddenly an expert; no matter how dysfunctional, inhumane and destructive their ideas are.
Anyone who suggests a paired programming solution is right, and answering the wrong question. Unless/until we return to a covid-like market the process will never be optimized for the candidate, and this is just too expensive an approach for employers. In this market I think the answer is hire less.
If using AI is cheating then one solution as the author mentions is have the interview take place at an office but I'm surprised another approach isn't more readily available: having the candidate the the test remotely at a trusted 3rd party location.
I just ask to share a text editor and write down my questions. Its critical anyway because often then not its not always clear for tech questions what exactly i asked (linux command for example).
This blocks their screen too.
and yes we do know very soon if you look somewere else, take time or rephrase the question to get more time.
If you able to fake it, at that point you should just get th ejob anyway :P
Interestingly I find AI is actually better at that kind of CS whiteboard question (implementing a binary search tree) than that "connecting middlewares to API" type task. Or at least, it's more straightforward to apply the AI, when you want a set of functions with clearly defined interfaces written - rather than making a bunch of changes across existing files.
I’ve wondered about the kind of person who starts white boarding with the pros and cons of several AI offerings. As if, confronted with a problem domain, they are choosing or hiring AI before architecture. As an interviewer, how should I adapt my questions? Something like, “How would you prompt it to add fuzzing?” “How would you prompt it to describe how each change might affect our stack?”
To test their inherent thinking skills and base knowledge.
There's a huge difference between occasionally looking up something, and practically leaning on it. Ironically, the mass degradation of search engine result quality within the past ~decade has made it much harder for people to do the latter, and when they do, it shows much more clearly.
I've been considering using a second webcam stream focused on my screen just to assure hiring managers that I don't have ChatGPT on my screen, or anywhere else. Kind of like chess players do it sometimes on online tournaments. I've been hearing people complain about cheating a lot.
I've been interviewing a bunch of developers the past year or so, and this:
> Architectural interviews are likely safe for a few years yet. From talking to people who have run these, it’s evident that someone is using AI. They often stop with long pauses, do not quite explain things succinctly, and do not understand the questions well enough to prompt the correct answer. As AI gets better (and faster), this will likely follow the same fate as the rest but I would give it some years yet.
Completely matches my experience. I don't do leet code BS, just "let's have a talk". I ask you questions about things you tell me you know about, and things I expect of someone at the level you're selling yourself at. The longest it's taken me to detect one of these scumbags was 15 minutes, and an extra 5 minutes to make sure.
Some of them make mistakes that are beyond stupid, like identity theft of someone who was born, raised and graduated in a country whose main language they cannot speak.
The smartest ones either do not know when to stop answering your questions with perfect answers (they just do not know what they're supposed to not know), or fumble their delivery and end up looking like unauthentic puppets. You just keep grinding them until you catch em.
I'm sure it's not infallible, but that's inherent to hiring. The only problem with this is cost, you're going to need a senior+ dev running the interview, and IME most are not happy to do so. But this might just be what the price of admission for running a hiring pipeline for software devs is nowadays. Heck, now feels like a good time to start a recruitment process outsourcing biz focused on the software industry.
I think this approach is not very favoured by hacker news but it's also what I prefer. It's so much easier to quickly gauge a minimum level of the basic programming knowledge and other sw knowledge by just asking some simple directed questions.
I once got a guy that claimed to have implemented multiple default HTTP JSON REST APIs and somehow had never:
- tested his API with JSON payloads, serialise serialise
- never queried his APIs manually or semi automatically (no knowledge of curl, postman or anything similar)
I never understood why Big Tech never setup contracts with all the SAT and ACT test centers across the country. Even before Zoom with Codepads it would have made sense for the recruiters to send potential candidates to a test center to do a pre assessment rather than waste time with engineers sitting on prescreen calls all day.
What exactly are you suggesting here? A standardized test that applies to all your job applications? Or, a candidate having to drive to a test center for every company they apply to? Or something else?
the idea is sound. create a basic standardized test targeted at tech/engineering jobs. not actually SAT -- operated by a vendor like The College Board. There are plenty of standardized test operators
When I'm interviewing, I'm putting about 30% of the weight towards "would I enjoy working with this person on a daily basis?", but in the context of technical discussions. Standardized testing won't be able to replicate it.
Discriminate against... a personality that will negatively impact the team dynamics? It's not that easy, to be honest, as every team has its own requirements.
A stereotypical Asian interviewing a stereotypical German might find the German rude in some interactions. While another German interviewer would find it being frank.
Interviews based on personal feelings have hidden biases not even the interviewer is aware of.
Here's another question - stereotypical Japanese interviewer, interviewing, back-to-back, a stereotypical Indian and a stereotypical German for the role. Both are capable and equally technically proficient. How do you choose, other than looking at the team you're hiring for, and thinking how the person would fit in?
You just send out a generic decline, and document it as there's a better candidate fit for the role.
I'm not sure if you guys have been in charge of hiring, but there's no real alternative. In my most recent experience, we had one open position, and after interviewing 10 candidates, 3 of them were basically identical in terms of technical qualifications. How do you choose one over the other, other than the "vibes"? Anyone suggesting otherwise is either living in a weird alternate reality, or doesn't want to accept that working is a cooperative job and interpersonal relationships are very important.
There always will be exceptions for different type of roles and specializations, but that's not what I'm talking about.
If the candidate is a protected class and they are rejected for "cultural fit" it will be an easy case for EEOC to raise a discrimination case.
This is effectively how Harvard was rejecting Asian applicants. They created a "personal fit" / cultural fit quality that Asians scored low on . Supreme Court found this to be discrimination.
It doesn't matter if you are truly discriminating, it matters how well you have tangible evidence of the employee not meeting the qualifications for the role.
Interviewing fads are set by these large companies that have the problem of systematically evaluating many thousands of candidates. A standardized test is all they want. Then they could do the rest like college admissions, and at a fraction of the cost.
given the in-person interview is at the end of the funnel by a factor of 500-1000, standardized testing might even open up opportunity for under represented candidates.
Think of how poor the screening process is at the recruiter & CTS (left side) of the funnel, and how many false negatives there are .
If you could offer standardized test at that level, you may be able to keep viable candidates in the funnel longer.
> The truth is I don’t trust anyone else to run evals.
It's a common sentiment.
But compare https://www.cambridge.org/core/journals/judgment-and-decisio... . ("People predicting the future performance of college students state that interviewing the students aids prediction, although in fact the interviews make predictions less accurate.")
Standardized plus the ability for companies to do their own test after they pass the standard one. So go get prescreened at test center then use that test to apply for jobs. Company either flys you in for in-person or sends you back to test center to do live remote interview in controlled environment.
It's been tried and failed: Sun Microsystems pushed certifications in the 90s. Pass the test on some technology, get the certification. Then they studied performance. The result? More certifications implied a worse employee. The reason was the top performing employees had no time to study for the exams, but the managers of the bottom performing employees were happy to send them off to training and testing. And then the certification fad came mostly to an end.
That was something quite different that got tried. This would be more based on aptitude rather than knowledge.
Of course, they'd miss out on some good talent. But in the article where it shows the quote of someone getting rejected for not inverting a binary tree on a whiteboard, that doesn't seem like a terrible thing to test for.
I have done interviews with Karat which just outsources technical interviews to engineers elsewhere.
All of these technical interviews still suck though! I basically never code with someone watching me and find it very difficult to do in interviews. I also find it hard to find the time to actually practice this skill
Im revisiting technical interview prep because it has been awhile and it seems like a good time for a refresher, and it is striking how similar it all is to SAT and GMAT prep these days. A pretty cookie-cutter performance that is mostly about demonstrating that you have the time and means to properly prepare. Might as well just go the extra step at that point and have it be exactly like those standardized tests… take them once at a test center, get a score that is valid for a few years that you can just send in with your application…
That maybe end up happening. Send them to a test center to do their remote hacker rank interview. Doesn’t even need to be standardized. I don’t like it but it’s one of the lower friction options.
In the US and Canada, to get into university, there is, or at least used to be, a standard entrance exam or something close to it (the SAT in the US, OAC scores if you were from Ontario, Canada, etc..).
Additionally, Undergraduate programs in the US and Canada, at least used to, despite their varied reputation, have a pretty standard program.
Maybe things have deteriorated so far at the high school and university level that new standardized exams are needed. But we also have a plethora of verifiable certifications whose exams are held in independent test facilities.
I miss one option from the list of non-solutions the author presents there - ditch the idiotic whiteboard/"coding exercise" interview style. Voila, the AI (non)problem solved!
This sort of comp-sci style exam with quizzes and what not maybe somewhat helps when hiring junior with zero experience fresh out of school.
But why are people with 20+ years of easily verifiable experience (picking up a phone and asking for references is still a thing!) being asked to invert trees and implement stuff like quicksort or some contrived BS assignment the interviewer uses to boost their own ego but with zero relevance to the day to day job they will be doing?
Why are we still wasting time with this? Why is always the default the assumption there that the applicants are all crooked hochstaplers that are lying on their resumes?
99% of jobs come with probationary period anyway where the person can be fired on the spot without justification or any strings attached. That should be more than enough time to see whether the person knows their stuff or not after having passed one or two rounds of oral interviews.
It is good enough for literally every other job - except for software engineering. What makes us the special snowflakes that people are being asked to put up with this crap?
Never really liked leetbro interviews. Always reeked of
“SO YOU THINK YOU CAN CODE BRO? SHOW ME WHAT YOU GOT!”
The majority of my work over 10+ years of experience always relied on general problem solving and soft skills like collaborating with others. Not rote memorization of in order traversal.
> Tech interviews are one of the worst parts of the process and are pretty much universally hated by the people taking them.
True.
> One of the things we can do, however, is change the nature of the interviews themselves. Coding interviews today are quite basic, anywhere from FizzBuzz, to building a calculator. With AI assistants, we could expand this 10x and have people build complete applications. I think a single, longer interview (2 hours) that mixes architecture and coding will probably be the way to go.
"One of the things we can do, however, is change the nature of the interviews themselves. Coding interviews today are quite basic, anywhere from FizzBuzz, to building a calculator. With AI assistants, we could expand this 10x and have people build complete applications. I think a single, longer interview (2 hours) that mixes architecture and coding will probably be the way to go."
If that's where the things are going, I'm retraining to become a line cook at McDonalds.
The image below does sum it up but not in the way the author thinks.
Google wants to hire people who complete their hiring process. They're OK with missing out on some people who would be excellent but who can't/won't make it through their hiring process.
The mistake may lie in copying Google's hiring process.
None of this makes any sense. Why should I complete a tech test interview if I have 15 years of experience at X top firm? I would have done it already anyway.
I had a ‘principal engineer’ at last place who grinded leetcode for 100 days and still failed a leetcode interview. It’s utter nonsense.
A conversation with technical questions and topics should suffice. Hire fast and fire people.
LLMs killed busy work. Now people have to actually talk to each other and they're finding out that we've been imitating functionality instead being functional.
What a BS article. As they say, just do the interview in person. Problem solved. Not sure about the US but 99% of jobs here in Spain are hybrid or onsite ("presencial"), not fully remote.
They're acting like all jobs are remote and it's impossible to do an interview in person.
Also, does it really matter? If a person is good at using AI and manages to be good at creating code with that, is it really so much worse than a person that does it from the top of their head? I think we have to drop the idea that AI is going to go away. I know it's all overhyped right now but there is definitely something to it. I think it will be another tool in our toolboxes. Just like stackoverflow has been for ages (and that didn't kill interviews either).
Ah weird, here in Europe if you apply for a job in another city or country they won't fly you out there. You can come on your own dime and usually you wouldn't even tell them you don't live there yet (after all if you don't even live there, why would they bother with you, it's only extra hassle for them when you start). Probably C-suite roles are an exception to this. But they're an exception to pretty much everything. Roles with particular foreign languages (e.g. support) too but this is also an edge case.
Moving personnel between countries when they are already working for the company does happen. They did it for me. But at that point they already know what they have.
Show the remote candidate an AI's deficient answer to a well-asked question, and ask the candidate if they understand what exactly is wrong with the AI's assessment, or what the follow-up/rewritten prompt to the AI should be. Compile a library of such deficient chats with the AI.
It's a tricky subject, because what if people who use AI are just better together? And what if in a year from now, AI by itself is better? What's the point of hiring anyone? Perhaps this is the issue behind the problems being described, which might be mere symptoms. There are tons of very smart teams working on software that will basically replace the people you're hiring.
Or you can just given them a way to bypass all of that, and ask them about any significant project that the candidate did build (which is relevant to the job description, open or closed source that is released) or even open source contributions towards widely used and significant projects. (Not hello world, or demo projects, or README changes.)
Both scenarios are easily verifiable (can check that you released the project or if you made that commit or not) and in the case of open-source, the interviewer can lookup at how you code-review with others, and how you respond and reason about the code review comments of others all in public to see if you actually understand the patches you or another person submitted.
A conversation can be started around it and eliminates 95% of frauds. If the candidate cannot answer this, then no choice but give a leetcode / hackerrank hard challenge and interview them again to explain their solution and why.
A net positive to everyone and all it takes to qualify is to build something that you can point to or contribute to a significant open source project. Unlike Hackerrank which has now become a negative sum race to the bottom quest with rampant cheating thanks to LLMs.
After that, a simple whiteboard challenge and that is it.
This would be a nice interview for candidates who have open source contributions, but many who have day jobs do not. Or their open source code is 5 years old and not representative of their current skill set.
There is no shame in taking time off after leaving a job to develop or contribute to an open source project or two. The world would be a better place for it.
That is a lot like saying that a college degree is financially straining or outright impossible. In many respects, developing open source is a lot less straining, as there are no large fees, with the main expenses being living expenses. This is why it's important to live far below one's means when one does have a job.
Well the reason I don’t has nothing to do with shame and everything to do with time. I’m allocating my extra time to work, which (on a good day) makes my company money.
For a candidate who does have OS contributions, that’s great but most will not. And the more senior they are the less likely I would imagine.
If you already have a job, you don't strictly need a different one. If you really need one, it should be okay to quit the old one first, perhaps work on open source as needed, then get a new one. As it is often said, looking for a job is a full-time job.
I cannot emphasize this enough. Coding is the EASY part of writing software. You can teach someone to code in a couple of months. Interviews that focus on someone's ability to code are just dumb.
What you need to do is see how well they can design before writing software. What is their process for designing the software they make? Can they architect it correctly? How do they capture user's mental models? How do they deal with the many "tops" that software has?
No it didn't, you just need to stop asking questions an LLM can easily solve, most of those were probably terrible questions to begin with.
I can create a simple project with 20 files, where you would need to check almost all of them to understand the problem you need to solve, good luck feeding that into an LLM.
Maybe you have some sneaky script or IDE integration that does this for you, fine, I'll just generate a class with 200 useless fields to exhaust your LLM's context length.
Or I can just share my screen and ask you to help me debug an issue.
I know nobody likes doing tech interviews but how has AI killed it ? Anyways you do want to know basics of computer science, it is a helpful thing to know if you ever want to progress beyond CRUD shitshovelling.
Also wtf is inverting a binary tree ? Like doing a "bottom-view". That shit is easy.
The best interview process I've ever been a part of involved pair programming with the person for a couple hours, after doing the tech screening having a phone call with a member of the team. You never failed to know within a few minutes whether the person could do the job, and be a good coworker. This process worked so well, it created the best team, most productive team I've worked on in 20+ years in the industry, despite that company's other dysfunctions.
The problem with it is the same curse that has rotted so much of software culture—the need for a scalable process with high throughput. "We need to run through hundreds of candidates per position, not a half dozen, are you crazy? It doesn't matter if the net result is better, it's the metrics along the way that matter!"
I dislike pair programming interviews - as they currently exist - because they usually feel like a time-crunched exam. You don't realistically have the freedom to actually think as you would in actual pair programming. i.e. if you wag your tail chasing a bad end for 15 mins, this is a fail in an interview, but it's pretty realistic of real life work and entirely a non-problem. It's probably even good to test for at interview: how does a person work when they aren't working with an oracle that already knows the answer (ie: the interviewer)?
Pair programming with the person for a couple hours, maybe even on an actual feature, would probably work, assuming the candidate is compensated for their time. I can imagine it'd especially work for teams working on open source projects (Sentry, Zed, etc). Might not be as workable for companies whose work is entirely closed source.
Indeed, the other problem is what you mention: it doesn't scale to high throughput.
> i.e. if you wag your tail chasing a bad end for 15 mins, this is a fail in an interview
In all pair programming interviews I have run (which I will admit have been only a few) I would fail myself as an interviewer if I was not able to guide the interviewee away from a dead end within 15 minutes.
If the candidate wasn't able to understand the hints I was giving them, or just kept driving forward, then they would fail.
Exactly! Who calls "researching how to build X, but then letting my pair-programming partner fall down a rabbit hole so I can feel superior" "pair programming".
Way too many career and remuneration focused techbros. Often driven by bullshit management and stack ranking style promotion/firing pathways.
> I would fail myself as an interviewer
You are not most interviewers, alas.
That's definitely up to the interviewer, in which a lot of discretion and trust has been placed. I think a lot of it also comes down to the culture of the company—whether they're cutthroat or supportive. As you get better people into the company, hopefully this improves over time. I know that when we did it, it was never about nailing it on the first try, it was literally about proving you knew how to program and were not an asshole. So, not the equivalent of reversing a binary tree on a whiteboard. The kinds of problems we worked on in the interviews weren't leetcode type problems, they were real tickets from our current project. Sometimes it was just doing stuff like making a new component or closing a bug, but those were the things we really did, so it felt like a better test.
Hiring doesn't scale, period; deal with it.
Exactly. People go like “the ideal way to interview is <whatever they themselves are the best at>”. Pair programming interviews suck and don’t scale, just like every other alternate way of hiring.
IMO interviewing is the biggest bottleneck and if interviewing was decoupled from hiring then it wouldn't be a problem. But this requires a Guild-like organization to manage interviewing/vetting and for companies to use said Guild for hiring. The companies could then do a single team culture meeting (if they wanted) before hiring.
I wish I could remember who, but there is a company out there who conducts tech interviews to create a pool of candidates for their customers. Pretty sure there was a post here about it, but it's lost in my ocean of unread favorites.
Triplebyte is the company.
You still interview with the end companies, but technical interviews aren’t given.
I personally think outsourcing hiring is the worst idea ever, and completely lose confidence in companies that do.
They’re outsourcing technical interviews, not the same thing as hiring.
(Update: technically I would guess they're only outsourcing the first few rounds. Presumably their customers will conduct further interviews.)
What if the best candidates have already been filtered out by then?
Create a scalable, practical interview process where the result is reliably the best candidate, and you'll take over the world.
At this point in my career I no longer even believe that "best candidate" is a meaningful description except perhaps for highly specialized roles. We're all stumbling blindly in the dark.
For companies that successfully scale their team size, it literally did scale, right? I think you mean that hiring is very difficult to scale.
Define successfully, I'm pretty sure they could have been a lot more successful by giving hiring the attention it deserves.
Exactly. Its almost like optimising for finding your best possible match for marriage. You don't go over a billion prospects, you choose from the ones locally available to you, as they come.
You're right, premature optimization is exactly what it is.
it isn't about scale. It is about core principle of the tech hiring - all the companies hire only the best. Not only it is impossible to scale, it is plain impossible. Even if all the companies hired only "above the average" it would still be a pretty tall order :)
There is no one-size-fits-all for software developers. Different companies are looking to hire people at different experience/pay levels, with different skill sets, etc. Just as with dating, most companies are also going to have some understanding of their own "attractiveness", and not try to date out of their league.
FAANG companies offering industry leading compensation packages and prestige are in a position to be able to hold out for the best (even if their interviewing practices may fail to achieve that), but most companies are just looking for someone that checks the boxes and seems like a good fit.
Employers trying to hire for fit and culture has resulted in the most inhumane and counterproductive processes I've witnessed. If that's what you really want to do, let the person work for at least a month in the team.
Provisional hires (which often exist in theory but they're pretty much a formality in general) don't work for the most part. Lot of overhead for the company. And in many cases you're asking for the candidate to quit a job and possibly relocate on the possibility they'll get a new position assuming that they click in a short time interval.
Who said anything about relocation? That has to be a tiny percentage of hires.
Relocation used to be pretty common for professional jobs. Don't know about today when there's more remote work. And maybe companies aren't as willing to pay for in general.
So, accept the overhead as the cost of hiring the right people?
There's even more overhead on the people being provisionally hired.
Yes, sometimes things just don't work out. But, if someone quits a job and maybe relocates, that's a big personal cost. It's just the way things work in some limited contexts (e.g. professional sports) but it's not and shouldn't be the norm.
I suppose you can give a huge sign-on bonus with no claw-back provision, but that's never going to happen in most cases.
I'm fully convinced the way to make better hires is to invest more, which will be more expensive. Which wouldn't be a problem unless we expected something else. It starts with quitting pretending the current process is working, or even close to optimal.
Has hiring ever really worked, anywhere? Especially as roles and need evolve? I guess you could argue that it sort of did, apropos of a play I saw last night on the astronaut program--and maybe the military in many cases more broadly.
But, in many cases, I'm not sure how I, as a candidate for a tech job, would feel about a company offering me $200K--no strings attached--with the proviso that I statistically only had a 25% chance of making it through the next 6 months. (And is that really long enough anyway?)
There are tournament-style professions. But I'm not convinced most professional jobs are or should be among them in general.
My first startup did one interview per person and then a trial period, all good.
>Has hiring ever really worked, anywhere?
yes. Best place i worked at - we hired only by internal references and only people from our University. Up until the company grew around 200 people. We didn't do technical interviews, just a short talk. And we were among top employers, including salary-wise.
When you have high trust a lot of other processes become unnecessary. When that trust is broken, and surely a lot of grifters BSed their ways into jobs, that’s when all kinds of barriers were added.
in the end it did not even achieve that. Those who spent 6 months at last job practicing leet code were not necessarily the best of the best
> i.e. if you wag your tail chasing a bad end for 15 mins, this is a fail in an interview
That’s an assumption. Perhaps following a dead end for a while, realizing it, pivoting, etc is a valuable, positive, signal?
I agree. But what I mean is: that's not how it's perceived in the current interview structure, which lasts maybe 45 minutes or so. Ultimately, going down a dead end means you'd now have 30 minutes to find the right solution and code it up. So the oracle (the interviewer) would probably help you realise sooner that it's a bad idea, so you don't waste your time. That's assuming they know the problem and solution well; if they don't, you'll just lose them and burn through your time.
In a 2 hour pair programming session on an 'unsolved' problem (like an open issue / minor bug / minor feature in a public repo), yes, it will likely not matter if you tried a bad idea, and would both be more realistic and a positive signal.
this is an interviewer problem, unless the candidate is totally silent. A candidate that can't ask questions isn't going to succeed, anyway.
If the candidate will communicate with me, I will offer them a LOT of guidance. It is still very, very easy to tell who knows what they are doing and who does not. You give them a basic but domain-specific task, you give them whatever extra context they need to do it, and you watch them hammer out the code.
It should be a task that is sufficiently familiar to the person applying to the role that they do not need to do -much- looking at docs, and as the interviewer you should be prepared to help them quickly find the docs you already know they will need -- you designed the task after all -- so that they don't waste time with that.
What's important is that they ask for docs when they need them, and that they can understand them, quickly, and use them. It will be obvious if they are using AI because of how long things will take. It will be obvious if they don't know when to reach for documentation, and it will be obvious if they cannot understand the documentation.
Then, they should write a test for their solution. This weeds out 95% of candidates. Talk to the other 5% and you'll find someone who can both actually write code and also discuss design.
A lot of good people will close-up and not voluntarily ask you any question if you put them under pressure. (What they automatically are on that exercise.)
What is not to say that you are making anything wrong. But watch for bias there.
> A lot of good people will close-up and not voluntarily ask you any question if you put them under pressure.
This sounds like the kind engineer who won't push back or ask for clarification on unclear requirements - and happily spend a month solving a problem the business doesn't have.
So maybe seeing that at interview time is a good thing?
Might not mean the candidate doesn't fit - but can clarify what kind of roles would work?
> So maybe seeing that at interview time is a good thing?
Perhaps, but an interview is a fundamentally different environment than day to day work.
There's no way to solve the interview/interviewee problem though, the whole thing is impossibly fucked and is going to have false positives/negatives no matter what.
I do 1hr pair programming interviews for my company and you have to strike a balance between letting candidates think through the problem even when you think it won't work (to see their thought process and maybe be surprised at their approach working/see how quickly they can self-correct) and keeping them on track so that the interview still provides a good signal for candidates who are less familiar with that specific task/stack.
I'm also not actually testing for pair programming ability directly, moreso ability to complete practical tasks / work in a specific area, collaborate, and communicate. If you choose a problem that is general/expandable enough that good candidates for the position are unlikely to go down bad rabbit holes (eg for a senior fullstack role, create a minimal frontend and api server that talk to each other) it works just fine. Actually with these kinds of problems it's kind of good if your candidates end up "studying" them like with leetcode, because it means they are just learning how to do the things that they'll do on the job.
> maybe even on an actual feature
I don't think this would work unless the feature were entirely self-contained. If your workaround is to give the candidate an OSS project they need to study beforehand, I think that would bias candidates' performance in ways that aren't aligned with who you want to hire (eg how desperate are they for the role and how much time outside of work are they willing to put into your interview).
Another problem is it is difficult to compare candidates whose interviews involved working on completely different problems.
> if you wag your tail chasing a bad end for 15 mins, this is a fail in an interview
Eh, if it's a reasonable bad end and you communicate through it, I wouldn't see it as a fail. Particularly if it's something I would have tried, too. (If it's something I wouldn't have thought of and it's a good idea, you're hired.)
I did a couple of rounds of this with my manager as the interviewer. Personally I really liked the process, and the feedback I got from the candidates was positive (but then again it always would be).
What worked well for me was that I made it very clear to my manager, a man who I trust, that I would not be able to provide him with a boolean pass/fail result. I couldn't provide him any objective measure of their ability or performance. What I could do was hang out with the canditates for an hour while we discussed some concepts I thought were important in my position. From that conversation I would be able to provide him a ranking along with a personal evaluation on whether I would personally like to work with the candidate.
I prepared some example problems that I worked for myself a bit. Then I went into the interviews with those problems and let them direct those same explorations of the problem. Some of them took me on detours I hadn't taken on my own. Some of them needed a little nudge at times. I never took specific notes, but just allowed my brain to get a natural impression of the person. I was there to get to know them, not administer an exam.
I feel like the whole experience worked super well. It felt casual, but also very telling. It was almost like a focused date. Afterwards I discussed my impression of the candidates with my manager to ensure the things I was weighing was somewhat commutable to what he desired.
All in all it was a very human process. It must have taken an enormous amount of trust from my manager to allow me the discretion to make a subjective judgment. I was extremely surprised at how clearly I was able to delineate the people, but also how that delineation shifted depending on which axis we evaluated. A simple pass/fail technical interview would have missed that image of a full person.
[dead]
I've (unfortunately) been interviewing the last two months and the main pattern that I've noticed is that a) big companies have terrible interview processes while b) small companies and startups are great at interviewing.
Big companies need to hire tons of people and interview even more so they need some sort of scalable process for it. An early stage startup can just ask you about your past projects and pair program with you for an hour.
I hear this all the time, but I have yet to experience it. It may be because the small companies that I interview with are all startups, but I have yet to be able to get a call back from any other kind of small company. And the startups I do interview with have a full FAANG interview loops.
There seems to be a weird selection bias that if you're FAANG or FAANG adjacent these small companies aren't interested.
At a former gig we had a newly hired ex-facebook employee give notice within a month because she didn't like that dev setup had bugs that devs themselves had to fix. At fb they obviously can spend millions of dollars for a whole team that ensures that working dev env is always a button click away, a startup (even a scaleup) usually can't afford to. This is just one example out of many I can tell...
And I’ve heard just as many horror stories about companies hiring from small companies that the engineers haven’t kept up with engineering, culture and practices and are coding like it’s 2004.
Also, those types of stories tend to pop up with any engineer who’s only worked at a single place.
My point isn’t that there’s not bad engineers at Facebook it’s that there’s bad engineers everywhere and filtering based on random signals like this is not useful.
The sad truth of hiring is that you can't afford to interview everyone, and have to keep in mind that prospective employees are showing you a different version of themselves than they'll actually bring to work.
Especially in a small company where your hiring manager may also be busy with development and sales, and not have an HR department to run the process for them, you're much better off interviewing candidates you are 50% sure of being a good fit vs 5%. Personally I prefer interviewing candidates coming from FAANG-ish companies and often make exceptions for candidates that demonstrate exceptional skill/interest, but when you can only interview 1-10% of your applicants you have to prioritize those who are likely to succeed at your company (keeping in mind implicit bias and such).
> filtering based on random signals like this is not useful.
In aggregate it most likely is useful for those companies.
> And I’ve heard just as many horror stories about companies hiring from small companies that the engineers haven’t kept up with engineering, culture and practices and are coding like it’s 2004.
Big cos can afford to onboard their engineers for months, sometimes years. Startups usually can not
>”There seems to be a weird selection bias that if you're FAANG or FAANG adjacent these small companies aren't interested.”
Many smaller companies have noticed that former and wannabe FAANGers are looking for FAANG-type jobs, and are not good fits in their niche. Small companies often have more uncertainty, fewer clear objectives, less structure, and often lower pay. They’re not a good substitute for megacorps.
And then there's people like me who have been at startups, midsize companies, tiny small businesses and FAANGs.
Not everyone at a FAANG is purely motivated by the amount of money that they can get.
I’m looking for a smaller company because I’m tired of the FAANG mentality personally.
The problem is that many smaller companies have hired people from FAANG, and had them quite after a short tenure, so they're unwilling to try their luck again, as it's just not worth it. You may be different, but they've heard that story before, and had it not work out.
> I’m tired of the FAANG mentality personally.
As someone that never had a desire nor ever made an attempt to work at any of those companies, do you mind elaborating on the mentality of such places?
I'm just your boring below-average to average dev, so I know I'm not cut for those types of places, but it never truly bothered me anyway. Any reason that I can personally think as to why I would work for such a company would either be due to my own egotistical desires or for monetary reasons, but those were never strong enough to actually compel me.
I am just mainly curious about two things:
1. Is working at those places all it's cracked up to be?
2. Assuming one had to work hard to get into such companies, was the juice worth the squeeze?
I've often wondered if one's experiences for these companies is often something akin to the old advice of, "Don't meet your heroes." In other words, was the conflicting dyad of expectations vs. reality present?
It has turned into something similar to what people in trading companies on Wall Street deal with. Constant grind, unrealistic expectations, and projects done in order to get a promotion instead of because it provides value to the customers or the business.
That said the amount that you make is insane some of the smartest engineers I’ve ever worked with have been at these companies and a lot of them have really strong engineering cultures, and standards.
The current work environment seems designed to use up bright young engineers, and burn them out within a few years. This is a significant shift from 15 years ago, where it was a much more sustainable place to be.
Yeah, that sounds hypercapitalist. At least I hope those engineers make enough money to retire when they burn out.
Unfortunately hedonic adaption often prevents that.
I could see that happening, especially given that they're young and feel powerful, nearly invincible. I experienced a similar attitude form younger folks outside of FANG, some kind of derision of experience and advice from people who have been around and choosing instead to obsolete it. After the first burnout may of them become a lot more thoughtful and humble. I may have been like that too when I was young, I don't remember being exactly like that but maybe I was just not aware. I still did respect experience and I still do to this day.
Sure, but in this context, your FAANG experience is a negative signal for people who don't know you well yet. It's unfortunate for you, but a genuine factor you now need to account for.
Your path through will probably look like having the luck of breaking in at one of these kinds of companies, and then staying for several years to demonstrate earnest commitment/fit while building a new network of connections, and then leveraging those connections to get more opportunities if it becomes necessary to do so. If you have connection from your previous non-FAANG work, that's probably your best route.
It won't happen overnight and you'll always be at a disadvantage when you find yourself applying through resume portals. Good luck!
I find this attitude baffling.
In my time at tier one companies I have worked with the best engineers I have come across in my entire career (even the worst engineers were more than competent) who were working on deep issues that could affect the revenue of the entire company because they’re laser focused on providing value to the business, instead of doing engineering for engineering’s sake. I have grown by far more in these kinds of roles than I have anywhere else because the kind of problems you encounter at such a high scale just don’t exist elsewhere. And most of them have been there for at least five years if not longer you don’t make those kind of contributions to accompany without a long tenure.
> In my time at tier one companies I have worked with the best engineers I have come across in my entire career
You’re throwing a giant red flag right here. First of all, FAANG isn’t “tier one” except to people who idolize these companies. More agile startups are trying to disrupt these dinosaurs and do not thing very highly of them. Many of us who have worked with FAANG and ex-FAANG engineers were not impressed.
I mean, I'm just sharing the practical ground truth of how a resume like yours effects recruiting in certain contexts.
Just like there are innumerable brilliant, effective engineers who would contribute tremendously to a FAANG but don't suit the modern interview funnel (leetcode, etc), smaller companies surely do miss out on strong, suitable FAANG engineers in anticipation of negative experiences they've had with others.
There are a lot of people who accumulate FAANG entries on their resume and many of them really don't suit smaller companies for a number of reasons.
Honestly, though while I'm only seeing a very narrow picture of you here, it sure sounds like you see these "tier one" companies as a desirable place to work, with prestigious colleagues and profound learning opportunities on high scale problems that just don't exist elsewhere, and surely for much more money. Are you sure you're really going to be happy somewhere else? Or might you get restless? That's precisely the kind of concern these smaller companies carry when seeing FAANG stuff on a resume, and it doesn't seem like it should be baffling that they would do.
I definitely used to. The work culture and attitudes, particularly of management, passed the breaking point for me a few years ago. I realized work was not my whole life nor did I aspire to that.
As I mentioned here (https://news.ycombinator.com/item?id=43121594)
Yup. You can check out of FAANG anytime you like, but you can never leave.
Was path dependency for careers always this bad?
I don’t feel like it was. Every role is hyper specific nowadays.
And most refused to look at anybody deviating from their ideal background in my experience.
>"And most refused to look at anybody deviating from their ideal background in my experience."
This is often because the culture of job-hopping for better pay every 18 months has eroded the willingness to pay for training or adaptation. Why pay for someone to learn if they're just gonna leave soon; the pre-trained person is a better deal if you'll have to pay to retain anyway.
Which was caused by cost cutting measures, MBA disease, in companies to begin with.
We’re just seeing the end of the cat and mouse struggle that’s been going on since the 60s. And massively accelerated in the 80s.
It’s unfortunate for companies though because they’re the ones that will lose out in the end when all the experienced people start retiring and they have no one to hire.
It’s an untenable position to not train people, period. There is no schooling you could go through that would educated junior dev to the level of a senior dev. And it’s the same for any other role. Experience is not optional.
I think the primary stimulus which creating the “job hopping culture” was actually the hot labor market for software developers. Other fields experienced real ‘cost-cutting’, without resulting in a lot of ‘job-hopping’.
I agree that this situation is undesirable, but it seems to be stable, somewhat like the result of repeated play of the prisoner’s dilemma.
That definitely massively accelerated it but you’re looking way too short term that’s only been in the last 10 to 15 years.
I agree that other industries are not YET at the point where software is , but you’re not looking hard enough if you don’t see the short tenures compared to the 25-30 years they used to have.
And yeah, it might be in an equilibrium now, but how long can it stay in an equilibrium? I’d guess at max 10 to 15 years.
It's a more mature industry.
I'm guessing the majority of people now in their 50s and 60s in computer-related careers had very eclectic jobs before settling down in computer-related stuff. After all, many never used computers at all until college or beyond.
My understanding is even in the early 2000s it was pretty much just firmware versus desktop software with a small niche for Mac developers.
Edit: my point was not that specialized software applications didn’t exist. It was that people were expected to be able to jump from stack to stack when they change roles in a way that has disappeared from modern job applications.
Plenty of server software being developed in the early 2000s. (Though minicomputers were mostly off the scene by then.)
Pretty much.
Well, and mainframes. And trading and financial systems. And numerical/scientific computing. And network services. And web sites and e-commerce. And flash, java applets, and browser plugins. And control systems. And operating systems and tooling. And cell phone applications. And games. And video/image/audio/music processing. etc etc
Oh, wait... maybe not!
So you’re saying that none of those roles could be cross hired in the early 2000s between any of the other roles?
That’s the point I was trying to make. Not that the software didn’t exist or people weren’t doing specialized applications.
It was probably about as hard to move between those domains now as it is today. Which is to say that it's pretty hard and needs some concerted, non-trivial effort in shaping your experience and how you present it before trying to make a transition, and often either some kind of inside reference to vouch for you or an employer that was especially hard up for candidates. Or else an employer that straddle multiple domains and actively supported internal transitions.
Depending on what you could bring attention to in your prior experience and the size/needs of the new orgs you seeking to move to, certain transitions were more feasible than others, but you could easily spend decades working in mind-numbing enterprise applications while wishing for opportunities in game development or trading or whatever and never get your resume so much as looked at. (And vice versa, even, for those who dreamed to "retire" into the supposed quiet of enterprise apps or government IT or whatever)
I basically agree with your edit. There was a lot more fluidity among roles and even just moving into computer roles from other engineering (and even non-engineering) fields. But that's not really what you wrote initially.
Fair
I've had a few good experiences with interviews at small companies and startups, so they do exist.
But I have also had really terrible experiences, similar to what you've mentioned. Sounds like you've just gotten unlucky and gotten the terrible ones.
Yeah, been there, done that; wannabe FAANGs are the worst.
At least those are typically honest about what they try to be and give a very clear signal, right at the interview time.
It's much worse when the interview gives you different vibes (and expectations) than the actual day-to-day work.
I've had both I've had ones that only tell you what the next interview looks like. Unless you specifically prior it out of the recruiter's hands.
The only way to test actual work is to do actual work, should be obvious but here we are.
It is obvious, but it’s also very time-consuming. You can’t solve ambiguous open ended problems in a set amount of time while being watched closely.
I'm not talking about being watched closely, I'm talking about regular work and then making a decision. You have to leave people space to do their thing if you want to see what they're capable of.
Sure, that would be even better. But how would that even look?
In the best case applicants needs to apply multiple companies. Companies need to interview multiple applicants and have a way to compare those applicants.
Those are the most basic constraints I can think of. How do you make that cost tens of hours for each round?
You would have to stop optimizing people to hell and back and start committing to a few at a time. Sounds like a really good idea to me.
That doesn’t answer the question.
For me as a job applicant even in the best case I would need to do 3 to 5 interview interviews. The same is true for companies in the best case it will take at least 3 to 5 interviews to find somebody. Are they supposed to have 3 to 5 temporary staff for weeks at a time?
How much time should that take per interview? How would somebody that currently has a job manage that kind of time commitment?
Yeah, you would need to change your expectations, I figured that much was obvious.
I felt like I was doing that with what I described.
What changes to expectations are you talking about?
You seem stuck on the idea that you need to verify an employee to hell and back before investing anything, that's not what I mean by committing. If they look like they might be a reasonable fit, give them a month to prove it, then you'll know for sure. One interview per person should be enough to make that decision.
What exactly does "scalable" mean here?
If a startup can spend 20 man-hours filling a single position, why can't a big company spend 1000 man-hours filling 50 positions?
In a small company, you can tell your buddy “just have a chat with the candidate and if you like them and you think they can do the job, hire them”.
If the person interviewing your candidates messes up, you’ll know soon enough. In a large company, the bad people will take over and your company is dying a slow death.
That approach doesn’t work on a large scale. Some interviewers are too nitpicky, elitist, others approve anyone who uses the same language as them for side projects. Some are racists, sexist, or have other kinds of biases. Some might have a crush on the candidate. Sometimes the interviewer thinks about their own task while they squeeze in an interview. In some countries, “undoing” a bad hire is hard, so they need to make sure that the candidate can work on any team (or at least on multiple teams reasonably well).
IMO for large companies it makes sense to standardize the interview process.
Also, in my opinion grinding leetcode is also a good personality check for FAANG hires: it shows the candidate can suck it up, study hundreds of hours, and do whatever they need to do to pass an arbitrary test, even if they themselves think it’s a broken process. The larger the company, the more this quality matters in candidates as they will need to deal with a lot of things they will probably not like.
> In a large company, the bad people will take over and your company is dying a slow death.
Why is this the assumption. I would rather say any big org. is converging towards the average talentwise by necessity. It is like Hawaii can't have 100 olympic level swimmers no matter the recruiting proces.
I'm quite conflicted on this. While I do not think one needs to remember/memorize a bunch of brainteasers or past computer scientists'/mathematician's PhD discoveries in order to build CRUD applications.
However, I do feel like there is perhaps some amount of truth to the thought behind the interview questions, no? As in, I would imagine someone that could invert a binary tree in 15 minutes on a whiteboard could probably learn React. However, I am not sure everyone that can learn React can invert a binary tree in 15 minutes on a whiteboard.
However, maybe I am projecting my own insecurities because I wish I could invert a binary tree in 15 minutes on a whiteboard as well as being able to solve all those other problems.
No, it's not actually about mastery of binary trees or other abstract computer science concepts.
leetcode is really a culture fit test and success points to the combination of "smarts", diligence, and conformity to some acceptable degree. It shows you have some baseline of familiarity with computing, can focus on an arbitrary task to pursue a goal, and will conform to an arbitrary process when it's asked of you.
Those are genuinely essential skills in an organization with 1000's of white collar professional workers.
More precise insight is gained when the test covers a binary tree inversion than that were it just some contrived logic puzzle like the LSAT or a critical reading exercise. The computer science bit does provide some signal and isn't completely arbitrary, but it's only a small part of what's being evaluated.
Among otherwise strong engineers, it tends to filter out the especially willful, prideful, independent, meandering, creative, and pragmatic ones. These personality types can be extremely valuable in some work environments and can still sometimes in through leetcode challenges, but spoil big bureuacratic systems like FAANG's when they're overrepresented.
The culture thing is huge. If you can’t find a few good leet code problems to enjoy, I don’t know if we are interested in the same field.
Having done a lot of interviews at least 70% of the time it’s filtering candidates who just aren’t very technical and haven’t studied computer science. The kind of people who hate reversing a linked list are the kind of people who haven’t touched a polynomial since high school.
It’s not unreasonable for a junior tech interview to expect you studied something like CS or EE. It’s a blessing that our field is open to all to give it a shot, but if that doesn’t describe you - you need to recognize that there is effort and study to fill that background.
"Leetcode" style tests were probably good when almost no one did it.
Now it selects for the very very very bad trait of high ambition and knowing to play the system.
But I'm talking about the idea of pair programming for an hour or two. Why can't the standardized test still involve that? What doesn't "scale" about it?
I think it’s easier to compare candidates who might have been interviewed by different people possibly weeks apart if you have a coding challenge… could they solve the problem we gave them, if yes how quickly and how well. Interviewers can write their notes, the session can be recorded and the committee can compare candidates relatively fairly.
Comparing candidates based on how they “vibe” with the interviewer during a pair programming session is a recipe for lawsuits and bad hires.
I’m just speculating here, I don’t have any significant hiring and interviewing experience
If you're worried about interviews weeks apart and cross-team testing, then the most that can hurt efficiency is making you do the test with the team they'll be imminently hired into, which means you're on par with the small company, nothing lost nothing gained.
Avoiding pair programming for the reason you listed sounds like lawyers getting in the way, not scaling. But yeah we'd need to hear someone say if that's actually happening.
> If a startup can spend 20 man-hours filling a single position, why can't a big company spend 1000 man-hours filling 50 positions?
Because big companies are run by bean counters and they also don't require the same kind of talent that is useful to startups. There's less competition for hyper-specialized seniors and middle of the pack generalists.
> If a startup can spend 20 man-hours filling a single position, why can't a big company spend 1000 man-hours filling 50 positions?
Huh. That’s actually a great question! I actually don’t know.
> small companies and startups are great at interviewing
Small companies have the benefit of the pressure to fill a role to get work done, the lack of bureaucratic baggage to "protect" the company from bad hires, and generally don't have enough staff to suffer empire-building.
Somewhere along the line the question changes from "can this candidate do the job that other people in this office are already doing?" to "can this candidate do the job of this imaginary archetype I've invented with seemingly impossible qualities".
"We need to run through hundreds of candidates per position, not a half dozen"
But you don't! You only need to find the first person who is good enough to do the job. You do not need to find the best person.
You need to run through hundreds of candidates to find someone marginally qualified. I am not exaggerating.
Do we have different definitions of "marginally qualified"? Idk, I feel I'm a decent engineer - I can certainly do whatever leetcode medium they throw at me, as much as this counts of anything - and can actually code, but I still get maybe 1 callback per 50 applications.
Does "marginally qualified" mean "Ivy League Competitive Programmer PhD" or something?
> Do we have different definitions of "marginally qualified"?
Not every candidate is an interview. I recently hired. I got 90 applications and from these, 80% were an instant "No". They didn't match the job description or had no permit to work in my country. I invited the rest. Simple interview, pair-program a dead simple App with a prebuilt skeleton with me with any framework of choice. Make one GET request, render it and realize one needed optimization which needs to be implemented. 90% (I'm not joking) of candidates failed the first task. In half an hour, they couldn't send a GET request and translate that into JSON. All were allowed to google, open any documentation they liked. Of all 18 who failed, 16 asked me if they could use an LLM for this task, which I denied.
Who are you interviewing? Do these people who can't do this have any experience at all?
The GP is still passing through hundreds of people, dozens of them capable, until he reaches somebody that convinces him of their competence. You were passed down because you weren't convincing enough.
Or maybe he is getting resumes from a channel that has been victim of machine-gun filling, and there indeed thousands of incompetent people posting resumes into every channel and just half a dozen real applicants.
TBF, I have no idea how to fix either one of those problems. Hiring is just completely broken.
I have only a resume to convince them. I have job experience at major companies, with examples of what I've built when there, personal projects I've made, a github link, a great GPA from a good school.
And I know similar Junior-mid people in the same boat. We can all do Fizzbuzz, we've all built things, and somehow we're not getting interviews, but people that can't do Fizzbuzz are.
Do thousands of incompetents also machine-gun apply to, say, mechanical engineering, accounting, marketing, HR, or finance gigs? Is it just tech?
Something isn't adding up.
> Do thousands of incompetents also machine-gun apply
Enough that it's HR or some automated software that first screens your application.
> Something isn't adding up.
Yes. The pipeline "posting > application > screening" is now completely broken. In your case it's quite possible it's HR and screening software that your resume is not convincing. I have been hoping for anecdotes and studies in this direction (from people who have access to the HR and/or software screening and are inclined to report) - but it's at least not common. What we do hear from, is tons of people who can program and who are not getting even first interviews.
Big companies and small need to agree on some standards and create qualifying exams that they will actually accept as proof of competence. Degrees somehow don't prove anything, experience doesn't, blah blah blah. It's exhausting to have to prove to interviewers that I'm not mentally disabled at every turn and it's a waste of time for everyone.
Create certifications that actually count for something and aren't just a blip on a resume that may tick a box, but will actually move you past technical trivia questions. I know some people have a deep repulsion to this and I think it would be fine to have a technical interview gauntlet for those that choose not to engage with any type of certification and a simplified interview format for those that have passed the prerequisite tests.
I don't care how long, rigorous, or ridiculous the tests are. Just agree on some effing standard.
Watch out for what you ask for. Plenty of big vendors have certification programs. ... And for some combinations of field and vendor, they are red flags - rather than pre-validation. That is, far too many applicants have the certification but do not have the grounding knowledge without which the certification is sort of useless - potentially more dangerous than validating.
Licensure. You're talking about licensure, not certifications. Other industries do it. Why doesn't software engineering?
>The best interview process I've ever been a part of involved pair programming with the person for a couple hours... You never failed to know within a few minutes whether the person could do the job
There is something funny about the "best interview process" taking "a couple hours" despite giving you the answer "within a few minutes". Seems like even the best process is a little broken.
Lightly ironic indeed! Though I’m not sure “broken” is exactly the word I’d choose.
I can only speak for myself, but I imagine myself as a candidate approaching a “couple of hours” project or relationship differently than I would a “few minutes” speed round. For that matter I can think of people I know professionally who I only know through their behavior “on stage” in structured and stylized meetings of a half hour or an hour—and I don’t feel like I have any sense at all of how they would be as day-to-day coworkers.
If we sat down to work together, you’d probably have a sense in the first few minutes of whether or not we were going to work out—but that would be contingent on us going into it with the attitude that we were working together as colleagues toward a goal.
That's mainly because there were multiple pairing sessions, and even if you knew the person was going to pass, there are still a couple more people who need to meet them, and a schedule to make sure they're available to do that. Plus due diligence, etc.
Nor am I saying it was a perfect system, just the best I've seen in terms of results.
The biggest victims of these non-scalable process is people without a good network. As an intl PhD student, I am that person.
So now I have this weird dynamic: I get interview calls only from FAANG companies, the ones with the manpower to do your so called "cursed" scalable interviews. But the smaller companies or startups, ones who are a far better fit for my specialized kills, never call me. You need to either "know someone" or be from a big school, or there is zero chance.
Some of us find the prospect of pairing with an unknown person in an unknown environment, and against the clock, to be very stressful.
Anecdote:
I've been interviewing recently and got through to the last round (of five...) with an interesting company. I knew the interview involved pairing, but I didn't expect: two people sitting behind me while I sat on a wobbly stool at a standing desk, trying to use a sticky wired mouse and a non-UK keyboard, and a very bright monitor (I normally use dark mode). They had a screen recorder running, and a radio was on in the next room.
I totally bombed it. I suspect just the two people behind me would have been enough though.
I would find trying to solve such problems with known people in known environments to be somewhat stressful too.
Pairing on something close to whatever real work they'd be doing, but familiar to the applicant is my favorite way to evaluate someone (e.g. choose a side project, pre agree adding a feature).
I don't care if someone uses modern tools (like AI assists), google, etc - e.g. "open book" - as that's the how they want to work. Evaluating their style of thinking / solving problems, comms, and output is the win.
Very few people doing this sort of interview (they tend to be our best, most empathetic developers) are likely to cut a multi-hour planned process short after a few minutes. It will eat at least an hour of their (very expensive & valuable) time.
Also how am I supposed to filter the 100's of AI-completed assessments? Who gets this opportunity?
We didn't do assessments (if by that you mean take home assignments). This was partly a solution to that, since nobody thought they were a good idea. If you mean the phone screen, I think that would be a problem, yep, but it wasn't an issue back in 2016. Having them pair would weed out cheaters, but we would have to figure out a way to weed them out during the screening, I agree.
We also did not require the employers doing the interview to be our most senior team members. They probably did it more often than most people, but often because they volunteered to do it. Anyone on the team would be part of the loop, which helped with scheduling. And, remember, we were working on actual tickets, so in a lot of cases it actually helped having the candidate there as a pairing partner.
For a little extra detail, the way we actually did it was to have 2-3 pairing sessions of up to 2 hours apiece. At the end of the day, all the team members who paired with the candidate had to give them the thumbs up.
Use another AI, of course!
Idk if I'm even being sarcastic here.
This. Interviewing for a sr dev position with a web app, backend stack is the bog standard java, spring, SQL abstracted away via JPA. We did a first screen, then the tech interview was two of their senior devs shoulder surfing me as I built a simple API. We chatted, I built, they asked questions, I defended my decisions (sometimes successfully, sometimes gracefully conceding defeat), they left knowing that I was who my resume said I was and the reminder that popped up in the middle of the interview to feed my sourdough starter showed them that I'm a culture fit.
I think you're onto something with that last paragraph but I want to try being a bit more generous with why things are the way they are. The question seems to be "When there are hundreds of applicants how do we give everyone a fair shake without hiring an entire team of devs who do nothing but interview?" From that perspective the intentionality is different and even sensible but the end product is likely to be the same. Even when someone is chasing a metric it's because someone else wants what's best and has decided that metric is a sensible way to make that happen. At the end of the day they really do want to hire the best candidate out of a pool whose size is extremely variable and that's challenging.
I work at a company which has 11 engineers and competes with companies with 100s. The hiring process was a screening call with the CTO to not waste the prospective team's time, then a call with 2 of my prospective colleagues to gauge competence and cultural fit. Since then I have been involved in hiring most of the team I work with now. The CTO is one of the most competent engineers I have ever met and he designed this process. He also has very high EQ. One of the points I sell to prospective hires is him as a person to work with, as well as our team. He has also flatly denied people I suggested before and that's fine.
I have been here 5 years now and I'm working with the most competent team I have ever worked with. My take away from this is that hiring doesn't need to be commoditised and scale, it just needs to find good people and give them an opportunity to show you that you do or don't want to work with them.
> You never failed to know within a few minutes whether the person could do the job
Then why spend a couple hours?
This is the exact time to use phrase “A people hire other A people, B people hire C people”
Additionally it’s rarely the hiring that makes a great team - it’s the long term commitment and investment in training.
I've been a proponent of pair programming since the early days of Agile, when it was still seen as part of extreme programming. Unfortunately, it’s not often employed in workplace settings.
With that said, would your perception of the interview remain positive if the outcome had been negative?
A common challenge across all interviews is a mismatch in personal dynamics, which can significantly impact the experience for both participants.
Consider a scenario where a senior developer, who prefers simplicity, is paired with a mid-level developer who is still captivated by complexity.
Or a "just start typing" person is with a "mull it over first" person. By the time I am typing code, I want to have 90% of it already completely worked out (at least till I type a "c.Lock()" and suddenly realize I hadn't considered thus and so synchronization issue.
Similarly, in general the best interviews I've ever been part of (either giving or being) turn into discussions where people's experience, opinions, and stories get aired (going both ways). You eventually get a good sense of each other and things get more relaxed when you both realize that you know what you're talking about (this is harder for Jr roles, though).
Being peppered with questions very rarely gives any insight.
For junior roles, you want to interview for intelligence and shall we say an interest in learning rather than specific skills.
Even for senior roles, that's what I want to interview for, although it is true at times a business case can be made for someone that is good at some specific complex skill and doesn't need to listen to other people to do ok work.
> person for a couple hours,
>You never failed to know within a few minutes whether the person could do the job
Did a misunderstood something or your best interview process is to multiple hours from someone when you've decided within minutes?
Pair programming on what problem though? I dont think many companies would want an outsider to work on their codebase.
I used to love getting to know the interviewer and doing things like that but IMO the market has shifted fundamentally on both ends for this to be effective anymore for most SaaS roles. This is anecdotal for US/Canada tech market over the past 10 years so YMMV.
Developers Side: Since developers don't have job security anymore (at least for those who work on common languages like Go, Python, Java and Typescript) they are better off learning and keeping in touch with leetcode and system design questions, looking for new opportunities and interviewing in "batch mode" when looking for a job. The idea is to clear as many interviews as possible using the same concepts, get in and make money asap before you get laid off. No incentive for collaboration or for fulfilling but esoteric stuff like Haskell and Scala. Career security > Job security.
Companies Side: On the other end software companies have less trust in developers staying long term so they want to make the interview process as quick and risk free as possible. In essence they are betting that by perusing 100s of resumes and hiring someone who seemingly knows CS concepts they can get some value out of them before they leave. Standardized tests/vetting > team fit.
TLDR; The art is gone from this job, its become akin to management consulting or investment banking. Quality and UX seems to be regressing across the board as a result.
>its become akin to management consulting or investment banking
Not sure how those are similar.
I meant the “grind”, short term profit mentality of SWE market has become similar to professionals in those fields, not that any of these fields are similar.
This is how my team hires and it’s incredible.
I think what makes it work is that our code pair is pretty low stakes. I was told that I didn’t have to finish the problem and I was free to use whatever tools or language I needed. They just wanted to see how I work and collaborate.
It's a super interesting approach, and if you put a strong filter before it (e.g. intense non-BS Q&A), the whole thing could be high-throughput.
Does your company operate the same way? I.e. is most, or at least a large chunk of engineering done as pair-programming?
Thats what we did, pair program on some real production code and tickets. This way the person could get a feel about what they potentially were walking into and you get a good idea of how they think and approach problems.
Beautifully articulated truth
Code reviews.
Teams are really sleeping on code reviews as an assessment tool. As in having the candidate review code.
A junior, mid, senior, staff are going to see very different things in the same codebase.
Not only that, as AI generated code becomes more common, teams might want to actively select for devs that can efficiently review code for quality and correctness.
I went through one interview with a YC company that had a first round code review. I enjoyed it so much that I ended up making a small open source app for teams that want to use code reviews: https://coderev.app (repo: https://github.com/CharlieDigital/coderev)
This is harder than it sounds, although I agree in a vacuum the idea is a good one.
So much value of the code review comes from having actual knowledge of the larger context. Mundane stuff like formatting quirks and obvious bad practices should be getting hoovered up by the linters anyways. But what someone new may *not* know is that this cruft is actually important for some arcane reason. Or that it's important that this specific line be super performant and that's why stylistically it's odd.
The real failure mode I worry about here is how much of this stuff becomes second nature to people on a team. They see it as "obvious" and forgot that it's actually nuance of their specific circumstances. So then a candidate comes in and misses something "obvious", well, here's the door.
You can do code review exercises without larger context.
An example from the interview: the code included a python web API and SQL schema. Some obvious points I noticed were no input validation, string concatenating for the database access (SQL injection), no input scrubbing (XSS), based on the call pattern there were some missing indices, a few bad data type choices (e.g. integer for user ID), a possible infinite loop in one case.
You might be thinking about it in the wrong way; what you want to see is that someone can spot these types of logic errors that either a human or AI copilot might produce regardless of the larger context.
The juniors will find formatting and obvious bad practices; the senior and staff will find the real gems. This format works really well for stratification.
> no input validation, string concatenating for the database access (SQL injection), no input scrubbing (XSS), based on the call pattern there were some missing indices, a few bad data type choices (e.g. integer for user ID), a possible infinite loop in one case
I'd say all this stuff is junior-level (maybe ~mid for things like user ID integers). It's just a checklist of "obvious bad practices", it doesn't require experience.
The senior stuff is much higher-level: domain modelling, code architecture, consistency guarantees, system resilience... system design in general.
You can do all of that in a code review; the point is that it actually allows for better stratification because you can incorporate different challenges in a reasonable time frame and without having to do take homes and get working environments (you'll end up reviewing their code anyways in a followup session).
You can do it in a real code review. I think his point was that you can't do stuff like "instead of loading a YAML file at runtime this should be generated during build time using the existing infrastructure we have here" type stuff.
But I'm not sure you really need to in a job interview. It's not like you can do that with any other interview method anyway - leetcode also doesn't really touch high level architecture type stuff, and take home problems are also too small (or they should be anyway!)
In my experience you only learn how good developers' architectural taste is by working with them for a long time.
Yes, ish.
In a previous job we did code review interviews. And went the route you said due to the problem I said. And yes, it's a lot better. But what also happened over time was that the bar slowly raised. Because over time the "harder" pieces of that session started to seem rote to interviewers, they became viewed as table stakes.
Mind you this is true of any interview scheme that has a problem solving component to it. I'm not saying that the code review style is extra bad here, just that it doesn't solve this problem.
I think the way to avoid the interviewer's expectations being raised over time is to write down some guidelines for what a successful candidate should be able to do. Even if you don't know how high to set the bar at the beginning, once you've hired someone you'll have at least one example of a good answer.
In theory, you can do code reviews without larger context if the code is carefully selected. Apparently, some companies think any badly written code from their code base can just be selected though.
It's not so hard. One of the interview stages I did somewhere well known used this.
Here's the neural net model your colleague sent you. They say it's meant to do ABC, but they found limitation XYZ. What is going on? What changes would you suggest and why?
Was actually a decent combined knowledge + code question.
There are so many interesting ways to use code reviews like subtly introducing defects and bugs and see if people can follow the logic, read the code, find where the reasoning comes up short.
I wrote up 7 general strategies for teams that are interested: https://coderev.app/blog/7-strategies-for-using-code-reviews...
I like the code review approach and tried it a few times when I was needed to do interviews.
The great thing about code reviews is that there are LOTS of ways people can improve code. You can start with the basics like can you make this code run at all (i.e. compile) and can you make it create the right output. And there's also more advanced improvements like how to make the code more performant, more maintainable, and less error-prone.
Also, the candidates can talk about their reasoning about why or why not they'd change the code they're reviewing.
For example, you'd probably view the candidates differently based on their responses to seeing a code sample with a global variable.
Poor: "Everything looks fine here"
Good: "Eliminate that global variable. We can do that by refactoring this function to..."
Better: "I see that there's a global variable here. Some say they're an anti-pattern, and that is true in most but not all cases. This one here may be ok if ..., but if not you'll need to..."
100% it is more conducive to a conversational exchange that actually gives you better insight into how a developer thinks much more so than leetcode.
Coding for me is an intensely focused activity and I work from home to boot so most of the time, I'm coding in complete silence. It's very awkward to be talking about my thought process while I'm coding, but not talking is just as awkward!
Some of the most interesting interviews that I felt like accurately assessed my skills (even non-live ones) where debugging and code review assessments. I didn't get offers from these cos later on because I failed the leetcodes they did later in the process but I felt the review interviews were a good way to be assessed.
Even more relevant now that we are in the age of code generation as the powertool for productivity.
I loved the idea of code reviews interviews, i've had several good ones, until yesterday when I had my first bad code review interview.
They asked me to review a function for a residential housing payment workflow, which I'm unfamiliar with. From an actual snippet of their bad production code (which has since been rewritten). In Go which I've never used (I've never professionally used the language that doesn't have error handling built-in, for example).
I had to spend more than half of my time asking questions to try and get enough context about Go error handling techniques, the abstractions they were using which we only had the import statements to and the way that the external system was structured to handle these requests to review the hundred lines of code they shared.
I was able to identify a bunch of things incidentally, like making all of the DB changes as part of a transaction so that we don't get inconsistent state or breaking up the function into sub functions, because the names were extremely long, but this was so far outside my area of expertise and comfort zone that I felt like shooting in the dark.
So just like any other interview style, they can be done very poorly.
Honestly this sounds like a successful "bad fit" signal (assuming that they work with go and payment systems mostly).
Language and domain experience are things id like to know after an interview process.
Honestly, it was also a red flag for me that they don’t actually know what they want and have bad communication between leadership and engineering. Prior to this interview I was already on the fence about them.
They don’t work mostly in Go. Even the interviewer said that he’s vaguely familiar with this area of the code, but he doesn’t work and Go. They work mostly in Kotlin and they explicitly are advertising for solid generalists.
I don't know. A cold code review on a codebase they never saw is not super informative about how the candidate would interact with you and the code once they're in known territory.
So yeah, I think it's the opposite: explicitly testing for their ability to read code is probably kinda important.
The demo video on your homepage looks great! If I may ask, what recording software did you use to create and edit the video?
Screen Studio on Mac.
(OBS Elements other times)
Is there a site where one could review some code and see what many others say about it and their experience level?
I guess it would degrade to stackoverflow-like poems eventually, but still interesting.
> Is there a site where one could review some code and see what many others say about it and their experience level?
https://codereview.stackexchange.com
This could be like rap genius for code!
Not that I know of!
It would be interesting, but I agree it would need to be content moderated to some extent.
github and issues page
I did this once and it was obvious the interviewer wanted me to point out his pet "gotcha." Not a great experience.
Yup, that's just one of the many ways to do a code-review interview wrong.
Each code sample should have multiple things wrong. The best people will find most (not necessarily all) of them. The mediocre will find a few.
Yeah it's really tempting when you discover an interesting fact to think "that would make an interesting interview question" and turn the interview into some kind of pub quiz. Happens with all forms of technical interview though. I mean 90% of leetcode questions are "this one weird trick".
Yep, I've done a lot of SQL interviews and it is always interesting to see the folks who've crash and burned at code review and killed it at writing individual queries and sometimes the unexpected, the opposite happened, the person would fly through a code review and do really subpar on writing it, a signal I usually took to mean that the person was nervous as hell in the interview.
The two folks who showed this behavior I hired anyway (they were contractors so nbd) and they were excellent hires, so I really love the code review approach for climbing up bloom's taxonomy.
Company A wants to hire an engineer, an AI could solve all their tech interview questions, so why not hire that AI instead?
There's very likely a real answer to that question, and that answer should shape the way that engineer should be assessed and hired.
For example, it could be that the company wants the engineer to do some kind of assessment whether a feature should be implemented at all, and if yes, in what way. Then you could, in an interview, give a bit of context and then ask the candidate to think out loud about an example feature request.
It seems to me the heart of the problem is that companies aren't very clear about what value the engineers add, and so they have trouble deciding whether a candidate could provide that value.
The even bigger challenge is that hiring experts in any domain requires domain knowledge, but hiring has been shifted to HR. They aren't experts in anything, and for some years they made do with formulaic approaches, but that doesn't cut it anymore. So now if your group wants to get it done, and done well, you have to get involved yourself, and it's a lot of work on top of your regular tasks. Maybe more work because HR is deeply involved.
>hiring has been shifted to HR
Well, unless you know sufficiently senior people. But I suspect that is a deeply unsatisfactory answer to many people in this forum.
My long term last, only technically-adjacent, job came through a combination of knowing execs, having gone to the same school as my ultimate manager, and knowing various other people involved. (And having a portfolio of public work.)
Personal networks only disadvantage those who have none.
I suspect many people who don't have strong networks for whatever reason resent that. To which you could probably tack on not having gone to the "right" schools or having a public portfolio.
also hard on introverts who already get punished in workplaces that promote ppl based on proximity and visiblity.
Self advocacy is part of the job.
And just because you’re an introvert doesn’t mean you are incapable of building soft skills. Talking to people is absolutely exhausting for me, but I force myself to do it and practice at it because I know it is important for my career.
Believe it or not, extroverts also have to develop professional skills, sometimes even things that don't come naturally to them.
No question. Yet, social connection seems worth 10x or 100x competence in any particular circumstance and the effects compound. There are some real benefits from and needs to be prosocial and socially competent but I've regularly seen social competent but technically incompetent people advance far over technically competent but less socially agenda driving people (that are nonetheless socially competent). This only gets worse at scale and as you progress.
I love coding and do it reliably well with joy but as my career has progressed I've struggled more and more with getting a company to let me work at a "low level" or to navigating what seem like sociopathic behaviors to really contribute at my capacity.
What do you mean by 'low level'?
Are you talking about a traditional Staff / Principal engineering role or something different?
Sorry for my lack of clarity, yes. Pure code and technical contribution, up to mentoring, as opposed to holding architecture summits, politicking, and the like. I've been pushed into management and socializing without regard to my willingness.
Well, they also promote people based on impact and, with rare exceptions, if you're holed up in a corner someplace you're probably not having a huge amount of impact.
Reality is that the ones quietly holed up in the corner are usually doing all the unsexy maintenance-type work that the extroverts don't want to do (because it's not sexy).
Nobody cares about that work... until it doesn't get done. And so, nobody doing it gets promoted.
Communicating the importance of your work is a professional skill.
And if it actually isn't very important, you should probably find something else to do or move on in some other way.
Or it gets outsourced which is what often happens.
If it was all about impact then ppl wouldn't be paying thousands of dollars to learn to play golf with their bosses. But you knew that already.
Is closing deals on the golf course even still a thing these days? I suppose it probably is in some circles but I haven't seen it in a couple of decades of tech industry life when it was more likely to be fun runs or skiing.
closing deals with your boss? what does that even mean.
Getting a promotion?
But my broader point was that golf course socializing seems like mostly a different world today, at least in my tech circles, relative to other venues.
As one, I have to say there's really nothing about being an introvert that prevents one from being affable and available. The idea is that human interaction does not boost the introvert's energy the way it does the extrovert's, not that it's impossible.
Dealing with people and communication can be learned.
I get it. By nature I was very much an introvert except for certain scenarios when I was in my comfort zone until at least my mid 30s. I was an only child, the stereotypical short, fat kid with a computer growing up in the 80s (still short, became a gym rat, part time fitness instructor and only stopped the latter as my other obligations became greater). Horrible dating life and a bad first marriage before turning 35 (happily remarried since then).
It became apparent that to get ahead in my career, “codez real gud” was going to limit my career. I slowly learned how to “act like I like people”.
But you can only add so much value to an organization typing on a keyboard. There is a reason that every single tech company promotes based on “impact”, “scope”, “dealing with ambiguity”. Those all require soft skills.
Well put. I'm an introvert, I can't do math, I won't travel, etc. are all things that some people claim as if it's the unchangeable nature of things. If that's their chosen path, so be it. But they should understand it will probably be pretty limiting because the world they live in.
Yes they are unchangeable. I've tried many many times to break out of it; it works for a while but i revert back to my base behavior.
We know what kinds of temperament a dog has within few months of it being a puppy ( and who the puppy's parents are). Why would it be different for humans.
Claim that Our temperaments ( and our likes/dislikes for travel) are all learnt is a bizzare blank slate claim that doesn't track with my life experience and what i've seen in the world.
> Why would it be different for humans.
Because animals typically live in a way more static and uneventful environments, and they have much more limited mental capabilities?
Humans (and other animals) aren't a completely black slate - but unlike most animals, humans have very complex societies that affect their behaviors throughout their entire lives. A few years in a different environment start to change people. Kids (with their still-growing brains) adapt faster, adults - not so much, but the traces will be evident. Move a not-too-fucked-up Russian to the Pacific Northwest, and they will eventually start to smile now and then.
Also, thanks to the language, humans can think things up even when alone, drive themselves crazy in all the weird ways, then overcome all that self-inflicted stress and possibly develop some behaviors as a result.
base temperament is irrelevant then?
My shy cat could've been a party animal like her sister :)
Of course it's relevant, by definition of "temperament". The question is how much of our (very complex) behavior is biologically based and independent of learning.
For a cat, it probably plays a significant role. Cat behaviors are complex but still much more simpler than humans. And changes are rare. Although I've heard of a "lazy" apathetic cat moving into a house with giant outdoor catio and becoming drastically more active, almost like a different kitty.
I'm not sure about humans - how much of our behavior is a true temperament and how much isn't despite tending to not change throughout one's life. I've seen introverts becoming eager activists after they went through some bad things, like war and prison. I've seen people who were jumpy and always nervous becoming relaxed after many years in safety.
I hated small talk traditionally. It was very much a learned trait. One of the best conversation openers is “what keeps you busy?” and then ask open ended questions.
Ask about their favorite travel destinations or even what are some interesting things about where they live.
On the other hand, step outside your comfort zone and try something different so you have something to talk about interesting.
https://tynan.com/letstalk/
You didn’t become a software developer overnight. You won’t become a great conversationalist over night either.
“How to Talk to Anyone”
https://www.amazon.com/How-Talk-Anyone-Success-Relationships...
(not an affiliate link)
People can certainly decide that certain activities aren't their thing.
I don't want to push to give presentations at international events is certainly a valid decision for any of a number of reasons.
So is preferring to spend more time on coding than managing/mentoring/etc.
But it all has consequences and some branches will lead to more promotions/money/etc. than others. And you may be perfectly fine with that. But go into with eyes wide open.
> But you can only add so much value to an organization typing on a keyboard.
In my understanding, non-junior software development jobs never were about typing on the keyboard. Senior software engineer is a fancy name for a problem solver, and code is just a specialized tool they can build to possibly achieve the goal. It always was about talking to stakeholders, figuring out what the heck they actually want today, how it fits with what they think they want tomorrow, learning more about those stakeholders so you can guess what they will think they want next week. Only then it's thinking about it all it for a while, and only after that it's getting to press the actual buttons.
But I'm not sure those things require "soft skills" aka - in my understanding - being a people person. For me, it was a very simple learning process - I (as a junior) coded something, a manager came next month and said I have to rewrite everything again because things have changed. I hated it, so I started to think how to possibly avoid or minimize it and optimize my own processes.
And in my mental model, it's not about people (save for tiny companies where a whole department/role is a single person, so I have to account for their mental chaos monkeys), it's all about business. That's why I wrote "stakeholders", intentionally dehumanizing (with no negative connotations) the model.
Thinking about engineering leads I know who aren't about leading huge teams, it's still about mentoring, talking to people, making connections, often talking to external audiences, etc.
I think a lot of that is "soft skills." Maybe not becoming a stereotypical sales person. But it's also not don't ever bug me and let me code.
The way that most tech companies define levels - yes I’m simplifying slightly. I’ll provide citations:
Junior - you are told what to do (business objective) and how to do it (technical).
Mid - you are told what to do (business objective). But you are expected to use your experience to figure out the “how”. You should be able to lead a decently complicated feature/epic/work stream either by yourself or with others and mentoring other juniors.
Senior - You are expected to lead major projects that involve multiple epics with multiple developers, talk to “the business”, disambiguate, deal with XYProblems, communicate trade offs between time, cost, meeting requirements, etc. Now you also start having to deal with cross team coordination.
Staff - cross team impact, dealing much more with business strategy and setting technical direction.
As an IC, the only things you have at your disposal are your relationships and reputation. Both require soft skills.
Leveling guidelines:
https://www.levels.fyi/blog/swe-level-framework.html
https://dropbox.github.io/dbx-career-framework/
I saw this at the big corporate (not faang/tech) place I work at. Engineers run and score interviews, but we don't make the final decision. That goes to HR and the hiring manager who usually has no technically background.
Yup, I have seen some really poor decisions as a result of this. I'm also curious - what will be the effect of AI assistance during behavior interviews, etc.
HR are experts in HR, which is to say they have a broader view of the institutional needs and legal requirements of hiring staffing than you do. It's always annoying when that clashes with your vision, but dismissing their entire domain is unlikely to help you avoid running into that dynamic again and again
> hiring has been shifted to HR
Not everywhere. At my company, HR owns the process but we -- the hiring tech team -- own the content of interviews and the outcomes.
I've never seen hiring completely in the domain of HR. HR filters incoming candidates and checks for culture fit etc, but technical competency is checked by engineers/ML folks. I can't imagine an HR person checking if someone understands neural networks.
HR involvement is unavoidable at big companies; and basics like "years of experience for payband" can cause issues. They fundamentally do not understand the job, but somehow have to ensure its not a biased hiring process.
Yes, and it is kind of necessary when hiring people from outside trusted networks. HR makes sure if people are who they say they are, background checks, and so on. Years of experience and so on are crude filters and should be bypassable by the team/hiring manager if the candidate meets the requirements. I know in large companies this can become political.
> Company A wants to hire an engineer, an AI could solve all their tech interview questions, so why not hire that AI instead?
Interview coding questions aren't like the day-to-day job, because of the nature of an interview.
In an hour-long interview, I have to be able to state the problem in a way the candidate can understand, within 10 minutes or so. We don't have time for a lecture on the intricacies of voucher calculation and global sales tax law.
It also has to be a problem that's solvable within about 40 minutes.
The problem needs to test the candidate meets the company's hiring bar - while also having enough nuance that there's an opportunity for absolutely great candidates to impress me.
And the problem has to be possible to state unambiguously. Can't have a candidate solving the problem, but failing the interview because there was a secret requirement and they failed to read my mind.
And of course, if we're doing it in person on a whiteboard (do people do that these days?) it has to be solvable without any reference to documentation.
> In an hour-long interview, I have to be able to state the problem in a way the candidate can understand, within 10 minutes or so. We don't have time for a lecture on the intricacies of voucher calculation and global sales tax law.
If you send me a rubric I can pre-load whatever you want to talk about. If you tell me what you're trying to build and what you need help with, I can show up with a game plan.
You need to make time for a conversation on the intricacies of voucher calculation and global sales tax law if you want to find people jazzed about the problem space.
> In an hour-long interview, I have to be able to state the problem in a way the candidate can understand, within 10 minutes or so. We don't have time for a lecture on the intricacies of voucher calculation and global sales tax law.
Proving if they are technically capable of a job seems rather silly. Look at their resume, look at their online works, ask them questions about it. Use probing questions to understand the depths of their knowledge. I don't get why we are over-engineering interviews. If I have 10+ years of experience with some proof through chatting that I am, in fact, a professional software engineer, isn't that enough?
Have you ever hired?
No, it's not enough. There are people out there who can talk great talk, and have great resume, but cannot do their actual job for some reason. Maybe they cannot read the code, maybe they cannot write the code, maybe they can write the code but not in the manner that keeps the rest of codebase working... I've had people like that on my team, it was miserable for all of us.
It is essential to see candidate actually write and debug code. It would be even better if we could see how the candidate deals with existing huge codebase, but sadly this kind of thing can't be easily done in a quick interview, and good candidates don't want trial periods.
>Interview coding questions aren't like the day-to-day job, because of the nature of an interview.
You have missed his point. If the interview questions are such that AI can solve them, they are the wrong questions being asked, by definition. Unless that company is trying to hire a robot, of course.
One of the best interviews I've encountered as a candidate wasn't exactly a pair programming session but it was similar. The interviewer pulled up a webpage of theirs and showed me a problem with it, and then asked how I would approach fixing it. We worked our way through many parts of their stack and while it was me driving most of the way we ended up having a number of interesting conversations that cropped up organically at various points. It was scheduled for an hour and the time actually flew by.
I felt like I got a good sense of what he would be like to work with and he got to see how I approached various problems. It avoided the live coding problems of needing to remember a bunch of syntax trivia on the spot and having to focus on a quick small solution, rather than a large scalable one that you need more often for actual work problems.
Problem is, company A doesn't need an engineer to solve those interview questions but real problems.
“Real problems” aren’t something that can be effectively discussed in the time span of an interview, so companies concoct unreal problems that are meant to be good indicators.
On that, these unreal questions/problems are decent proxies for general knowledge for humans, but not for AI. Humans don't have encyclopedic knowledge, so questions on a topic can do a decent job of indicating a person has the broader depth of knowledge in that topic and could bring that to bear in a job. An AI can answer all the questions but can't bring that to bear in a job.
WE saw this last year with all the "AI can now pass the bar exam" articles, but that doesn't lead to them being able to do anything approaching practicing law, because AI failure modes are not the same as humans and can't be tested the same way.
Really? How short are your interviews, and how big are these Real Problems such that you can't get a sense of how your candidate would start to tackle them?
The “real problems” most companies want people to help solve involve the evolution of products that last for years, involve repeated design discussions, in depth research, and applying retrospective learning. I don’t need someone that can just glue a Rails API together. If I did, I can literally just download that from the internet for free.
If my problems could be solved in the time span of an interview, why would I waste my time doing that interview instead of just solving it?
I don't see the issue here. Nobody expects candidates to build actual product during the interview. Having a (targeted, scope and time-limited) design discussion or giving your candidate some made-up context around an engineering cycle and then doing a retrospective with them are practical and useful ways to interview a candidate.
I'm also not sure what the alternative is? Just not hiring?
> Having a (targeted, scope and time-limited) design discussion or giving your candidate some made-up context around an engineering cycle and then doing a retrospective with them
You just described a contrived, “unreal” problem.
> I'm also not sure what the alternative is? Just not hiring?
The alternative is to come up with questions that are representative of skills related to “real problems”, as you just did, and use those instead. Unfortunately candidates consistently complain that such questions aren’t realistic.
I've had some success with just describing what we're doing and seeing what the candidates ask.
Mind, I work in very small companies and never had to give input for filling 10 positions at once... just one at a time.
I've tried this, but it becomes very hard to justify, with clarity, why it's a yes or no in the feedback, in a way that can be understood well as it passes through all of those up the chain that are involved with hiring.
And, I've also had people speak very well, doing great with the verbal explanation and questions, even good pseudo code, and then be unable to write a simple for loop, of any kind, in any language. These people also often have a resume full of short runs.
So, I structure mine around a, fixed, work related problem that lets me clearly justify the yes/no in a way that upper management can stomach, but then just bias my feedback a bit based on the "personal interpretation" things like what you describe (which I think are usually better indicators).
Also, resumes are 90% fiction, from what I've seen, especially from certain demographics (not allowed to perceive that though). I don't bother believing them or talking about them, unless there's time after.
> well as it passes through all of those up the chain that are involved with hiring
Yes, this mostly works in small organizations. I'm mostly in positions where I have to pass the feedback once, or at most twice up the chain.
OK I think we're on the same page, and just had a semantic issue.
This is the answer.
Let's not pretend 95% of companies are asking asinine interview questions (though I understand the reasons why) that LLMs can easily solve.
Let's go one step further: LLMs can't solve anything, but most interview questions are covered so much online that they'll parrot a passable answer.
Yeah, what you want is a General Intelligence that has learned the topic you care about. Google search returning an algorithm when you ask it doesn't mean that you shouldn't test candidates on that algorithm, since you still need a General Intelligence that knows it and not just the algorithm itself.
Tech interviews in general need to be overhauled, and if they were it’d be less likely that AI would be as helpful in the process to begin with (at least for LLMs in their current state).
Current LLMs can do some basic coding and stitch it together to form cool programs, but it struggles at good design work that scales. Design-focused interviews paired with soft-skill-focus is a better measure of how a dev will be in the workplace in general. Yet, most interviews are just “if you can solve this esoteric problem we don’t use at all at work, you are hired”. I’d take a bad solution with a good design over a good solution with a bad design any day, because the former is always easier to refactor and iterate on.
AI is not really good at that yet; it’s trained on a lot of public data that skews towards worse designs. It’s also not all that great at behaving like a human during code reviews; it agrees too much, is overly verbose, it hallucinates, etc.
I want to hire people who can be given some problem and will go off and work on it and come to me with questions when specs are unclear or there's some weird thing that cropped up. AI is 100% not that. You have to watch it like a 15 year old driver.
A company wants to hire someone to perform tasks X, Y and Z. It's difficult to cleanly evaluate someone's ability to do these tasks in a short amount of time, so they do their best to construct a task A which is easy to test, and such that most people who can do A can also do X, Y and Z.
Now someone comes along and builds a machine that can do A. It turns out that while for humans, A was a good indicator of X, Y and Z, for the machine it is not. A is easy for the machine, but X, Y and Z are still difficult.
This isn't a sign that the company was wrong to ask A, nor is it a sign that they could just hire the machine.
It's because coding interview questions aren't so much assessing job skills as much as they are thinly veiled IQ tests.
I think if it was socially acceptable they'd just do the latter.
Plenty of companies administer IQ tests. The reason everyone doesn't is that it doesn't work well.
Nothing works well but IQ tests predict job performance better than anything else.
Do you have a citation?
Schmidt et al., 2016 The Validity and Utility of Selection Methods in Personnel Psychology: Practical and Theoretical Implications of 100 Years of Research Findings
Also, an explanation for why more companies aren't doing them given their effectiveness? They're extremely easy to administer.
Possibly legal and reputational risks, considering some groups do badly on IQ tests.
No, this is a message board canard. IQ tests are used at a variety of large companies with deep pockets for discrimination settlements. If there were real legal risks, that wouldn't be the case.
There are real risks for companies without deep pockets (for settlements or public relations). People I know, responsible for hiring, have told me they won't use IQ tests because of how it would come across, so the concern at least exists but how widespread is the question.
Your citation addresses it. Less than 1% of employment lawsuits are about selection criteria and employers win over 90% of them. They suggest GMA tests are _more_ defensible than other approaches.
The most interesting thing in that paper is that years of experience performed so poorly. It’s in the lowest cohort. Worse than “interests” or more general “biographical data”.
There is still the reputational risk of using selection methods with widely known disparate outcomes. Other methods also have disparate outcomes, but most of the criticism is directed at IQ tests. I've heard "IQ tests are culturally biased" but never "work sample tests are culturally biased", and I'll guess that's the experience of most hiring managers too.
Have you ever heard or read a newspaper article, or can you cite an actual example of any company actually suffering reputation harm for administering iq tests? Your citation suggests candidates view GMA tests _favorably_.
Most hiring managers believe experience matters in hiring as well, perhaps that’s the belief that keeps them from using iq tests.
For what it’s worth, IQ tests are biased (see duyme’ adoptive studies for a drastic economic impact). That largely is orthogonal to if they are predictive in the ways your citation outlines.
Somehow these reputational risks accrue to the median hiring company, but not to the global brands like PepsiCo and Proctor & Gamble that do GMA testing already. I maintain: this is a message board trope.
Surely you can understand that the hiring needs and reputational risk of a tech startup and Pepsi are different?
Can you cite Pepsi suffering any reputational risk based on the practice?
[flagged]
A lot of companies have IQ like tests, in particular big consulting companies like McKinsey and so on.
McK's case interview is just as game-able as HackerRank style interviews. There are entire consulting clubs at many colleges that teach this exact interview style. It's true that it's harder (but not impossible) to use AI to help, but calling it an IQ-like test is true only as much as any other technical interview.
That being said, McK did create an entire game that they claim can't be studied for ahead of time. If the intention is to test true problem solving skills, then maybe that's roughly equivalent to a systems interview, which is hard(er) to cheat .
> That being said, McK did create an entire game that they claim can't be studied for ahead of time. If the intention is to test true problem solving skills, then maybe that's roughly equivalent to a systems interview, which is hard(er) to cheat .
Sure, right up until someone leaks it
Maybe, but they have hundreds! And new ones every quarter! Having done both systems and case interviews I can say that leaking is not as big a problem as with coding interviews. Sure you can memorize how to build Netflix/Zoom, or how to analyze P&L for a sandwich shop, but those interviews depend on interviewers throwing in complications or asking clarifying questions.
IQ tests are just as gamable.
And they're losing all but the worst candidates because of it, which explains a lot.
This is a great point. Though what if the answer is that the company can hire that AI to solve a significant fraction of its actual problems? People who do the assessments and decide what features should look like are often called managers (product, engineering, etc.).
For a while I’ve been skeptical that the rate of hiring of engineers would change significantly because of LLMs, but I’m starting to feel like maybe I’m wrong and it’s already changing and companies are looking toward AI to lower costs and require fewer humans. In that case they are probably still going to want people who are technically exceptional - maybe even more so - but are able and willing to create, integrate, and babysit AI generated code, and also do PM and EM style feature management.
If companies are slowing hiring due to AI, I would expect interviews to get worse before they get better.
> For example, it could be that the company wants the engineer to do some kind of assessment whether a feature should be implemented at all, and if yes, in what way. Then you could, in an interview, give a bit of context and then ask the candidate to think out loud about an example feature request.
So a Product Manager?
In most companies every engineer above a junior level is expected to pass features and bugfixes through their common sense filter and provide feedback. Product managers and designers aren't infallible and sometimes lack knowledge about the system or product that an engineer might have.
You can't just take requirements and churn out code without a critical eye at what you're doing.
Maybe.
Maybe now, or maybe in a year or two, AI coding tools will be good enough that a single semi-technical person can be Product Manager for a small product, and implement all the feature through AI/LLM tools.
Probably not for something of the complexity of Google Maps, but for a simpler website with some interactive elements, that could work.
But then, this was just an example. There can be lots of reasons that companies still need engineers, my point was that they need to think about these reasons, and then use these reasons to decide how to select their engineers.
[dead]
I was asked by an SME to code on a whiteboard for an interview (in 2005? I think?). I asked if I could have a computer, they said no. I asked if I would be using a whiteboard during my day-to-day. They said no. I asked why they used whiteboards, they said they were mimicking Google's best practice. That discussion went on for a good few minutes and by the end of it I was teetering on leaving because the fit wasn't good.
I agreed to do it as long as they understood that I felt it was a terrible way of assessing someone's ability to code. I was allowed to use any programming language because they knew them all (allegedly).
The solution was a pretty obvious bit-shift. So I wrote memory registers up on the board and did it in Motorola 68000 Assembler (because I had been doing a lot of it around that time), halfway through they stopped me and I said I'd be happy to do it again if they gave me a computer.
The offered me the job. I went elsewhere.
You should’ve asked them “do you also mimic google’s compensation?”
I work for a faang subsidiary. We pay well below average salary and equity. We finally got one nice perk, a very good 401k match. A few months later it was announced that the 401k match would be scaled back "to come in line with what our parent company offers". I thought about asking "will be getting salaries or equity in line with what our parent company offers?" but that would have been useless. Management doesn't care. I'm job hunting.
Oh man I needed that in the clip for like a dozen interviews a decade ago.
This zinger I have to remember for the next time someone tries this whiteboard BS on me!
"How many google shares do I get?"
> I was asked by an SME to code on a whiteboard for an interview (in 2005? I think?). I asked if I could have a computer, they said no. I asked if I would be using a whiteboard during my day-to-day. They said no. I asked why they used whiteboards, they said they were mimicking Google's best practice.
This looks more like a culture fit test than a coding test.
Yeah, very bad fit. Surprised they made an offer.
Folks getting mad about whiteboard interviews is a meme at this point. It misses the point. We CANT test you effectively on your programming skillbase. So we test on a more relevant job skill, like can you have a real conversation (with a whiteboard to help) about how to solve the problem.
It isn't that your interviewer knew all the languages, but that the language didn't matter.
I didn't get this until I was giving interviews. The instructions on how to give them are pretty clear. The goal isn't to "solve the puzzle" but instead to demonstrate you can reason about it effectively, communicate your knowledge and communicate as part of problem solving.
I know many interviewers also didn't get it, and it became just "do you know the trick to my puzzle". That pattern of failure is a good reason to deprecate white board interviews, not "I don't write on a whiteboard when i program in real life".
> We CANT test you effectively on your programming skillbase. So we test on a more relevant job skill, like can you have a real conversation (with a whiteboard to help) about how to solve the problem.
Except, that's not what happens. In basically every coding interview in my life, it's been a gauntlet: code this leetcode medium/hard problem while singing and tapdancing backwards. Screw up in any way -- or worse (and also commonly) miss the obscure trick that brings the solution to the next level of algorithmic complexity -- and your interview day is over. And it's only gotten worse over time, in that nowadays, interviewers start with the leetcode medium as the "warmup exercise". That's nuts.
It's not a one off. The people doing these interviews either don't know what they're supposed to be looking for, or they're at a big tech company and their mandate is to be a severe winnowing function.
> It isn't that your interviewer knew all the languages, but that the language didn't matter.
I've done enough programming interviews to know that using even a marginally exotic language (like, say, Ruby) will drastically reduce your success rate. You either use a language that your interviewer knows well, or you're adding a level of friction that will hurt you. Interviewers love to say that language doesn't matter, but in practice, if they can't know that you're not making up the syntax, then it dials up the skepticism level.
They generally do not know what they are looking for. They are generally untrained, and if they are trained, the training is probably all about using leetcode-type problems to give out interviews that are sufficiently similar that you can run stats on the results and call them "objective", which is exactly the thing we are all quite correctly complaining about. Which is perhaps anti-training.
The problem is that the business side wants to reduce it to an objective checklist, but you can't do that because of Goodhart's Law [1]. AI is throwing this problem into focus because it is basically capable of passing any objective checklist, with just a bit of human driving [2]. Interviews can not consist of "I'm going to ask a question and if you give me the objectively correct answer you get a point and if you do not give the objectively correct answer you do not". The risk of hiring someone who could give the objectively correct answers but couldn't program their way out of a wet paper bag, let alone do requirements elicitation in collaboration with other humans or architecture or risk analysis or any of the many other things that a real engineering job consists of, was already pretty high before AI.
But if interviewing is not a matter of saying the objectively correct things, a lot of people at all levels are just incapable of handling it after that. The Western philosophical mindset doesn't handle this sort of thing very well.
[1]: https://en.wikipedia.org/wiki/Goodhart%27s_law
[2]: Note this is not necessarily bad because "AI bad!", but, if all the human on the other end can offer me is that they can drive the AI, I don't need them. I can do it myself and/or hire any number of other such people. You need to bring something to the job other than the ability to drive an AI and you need to demonstrate whatever that is in the interview process. I can type what you tell me into a computer and then fail to comprehend the answer it gives is not a value-add.
> The Western philosophical mindset doesn't handle this sort of thing very well.
Mind elaborating on that?
It is a gross oversimplification but you can look at the Western mindset as being a reductionistic, "things are composed of their parts" sort of view, and the Eastern mindset as a holistic mindset where breaking things into their components also destroys the thing in the process.
The reality isn't so much "in between" as "both". There is a reason the West developed a lot of tech and the East, despite thousands of years of opportunity, didn't so much. But there is also a limit to the reductionistic viewpoint.
In this case, being told that the only way to hire a truly good developer is to make a holistic evaluation of a candidate, that you can not "reduce" it to a checklist because the very act of reducing it to a checklist invalidates the process, is something that a lot of Western sorts of people just can't process. How can something be effectively impossible to break into parts?
On the other hand, it is arguably a Western viewpoint that leads to the idea of Goodhart's law in the first place; the Eastern viewpoint tends to just say "things can't be reduced" and stop the investigation there.
This is highly stereotypical, of course, and should be considered as an extremely broad classification of types of philosophy, and not really associated directly with any individual humans who may happen to be physically located in the east or west. Further as I said I think the "correct" answer is neither one, nor the other, nor anything in between, but both, so I am not casting any shade on any country or culture per se. It is a useful, if broad, framework to understand things at a very, very high level.
When I joined my current team I found they had changed the technical test after I had interviewed but before I joined. A couple of friends also applied and got rejected because of this new test.
When I finally got in the door and joined the hiring effort I was appalled to find they’d implemented a leetcode-esque series of challenges with criteria such as “if the candidate doesn’t immediately identify and then use a stack then fail interview”. There were 7 more like this with increasingly harsh criteria.
I would not have passed.
> can you have a real conversation (with a whiteboard to help) about how to solve the problem
And do you frame the problem like that when giving interviews? Or the candidates are led to believe working code is expected?
Do I? yes. I also teach my students that the goal of an interview is to convince the interviewer you are a good candidate, not to answer the questions correctly. Sometimes they correlate. Give the customer what they need not what they asked for.
Do I see others doing so? sadly no.
I feel like a lot of the replies to my comment didn't read to the end, I agree the implementation is bad. The whiteboard just isn't actually the problem. The interviewers are.
Unless they change mentality to "did this candidate show me the skills i am looking for" instead of "did they solve puzzle" the method doesn't matter.
The replies are addressing the reality of the interview landscape that fails to live up to your theory of how whiteboarding interviews should be.
It's all well and good that you and other "wise interviewer" commenters on HN actually grok what the point of interviews are, but you are unicorns in the landscape.
I don't think you made it to the last paragraph either:
> I know many interviewers also didn't get it, and it became just "do you know the trick to my puzzle". That pattern of failure is a good reason to deprecate white board interviews, not "I don't write on a whiteboard when i program in real life".
Nope, it was directed at your last paragraph.
> The goal isn't to "solve the puzzle" but instead to demonstrate you can reason about it effectively, communicate your knowledge and communicate as part of problem solving.
...while being closely monitored in a high-stakes performance in front of an audience of strangers judging them critically.
That’s a skill you do need at Google if you’re going to survive. At least nowadays.
Except that 99% of engineers aren't being hired by Google nor being paid on comparable levels.
So why is Google relevant to this in any way?
> Except that 99% of engineers aren't being hired by Google nor being paid on comparable levels.
Sucks for you, then. Why are you on a thread about Google-style interviews?
> Why are you on a thread about Google-style interviews?
For the same reason you wrote "Google-style". Because this thread is specifically about those interviews happening not at Google.
Oh, maybe you misunderstood their question. When they suggested Google wasn't relevant, they meant the company culture at Google itself because that's what you were talking about.
Perhaps. I'd even say it's part of what is taught as part of a PhD.
But if someone was ready for your exact question by having the right interview practice/experience, or they just don't care about your job so there's no stakes. Then you still aren't measuring what you think you are.
> So we test on a more relevant job skill, like can you have a real conversation (with a whiteboard to help) about how to solve the problem.
Everybody says that, but reality is they don't imho. If you don't pass the pet question quiz "they don't know how to program" or are a "faker", etc.
I've seen this over and over and if you want to test a real conversation you can ask about their experience. (I realize the challenge with that is young interviewers aren't able to do that very well with more experienced people.)
+1 to all this. It still surprises me how many people, even after being in the industry for years, think the goal of any interview is to “write the best code” or “get the right answer”.
What I want to know from an interview is if you can be presented an abstract problem and collaboratively work with others on it. After that, getting the “right” answer to my contrived interview question is barely even icing on the cake.
If you complain about having to have a discussion about how to solve the problem, I no longer care about actually solving the problem, because you’ve already failed the test.
I think you're severely underestimating how much just about every software company has bought into the FAANG philosophy, and how many candidates they get who can answer those questions correctly.
Yes if you don't communicate clearly, you will get points deducted. But if you can't answer the question nearly perfectly, its basically an immediate fail.
Unfortunately I used to think this was the main purpose of the interview as well, but have been proven wrong time and time again.
The only thing that matters in most places is getting to the optimal solution quickly. It doesn't matter if you explain your thought process or ask clarifying questions, just get to the solution and answer the time and space complexity correctly and you pass.
Like others have said I think this is a symptom of the sheer number of people applying and needing to go through the process, there is no time for nuance or evaluating people on if you would actually like to work with them or not.
Are there people who still aren't aware that FAANGs developed this kind of thing to bypass H1-B regulations?
> The offered me the job. I went elsewhere.
I am so happy that you did this. We vote with our feet and sadly, too many tech folks are unwilling to use their power or have golden handcuff tunnel vision.
>I was allowed to use any programming language because they knew them all (allegedly).
After 30 years of doing this, I find that typically the people who claim to know a lot often know very little. They're insecure in their ability so much that they've tricked themselves into not learning anything.
Take a bow.
And my axe…
>I was allowed to use any programming language because they knew them all (allegedly). brainfuck time
Hehe, I have to remember to bring one of my custom Forths to the next interview.
As an interviewer I’d just skip the questions and talk about your Forth haha
Even better :)
I have yet to come across an interviewer who has a clue about anything that interests me.
2005? You were in the right.
Today? Now that's when it is tricky. How can we know you are not one of these prompt "engineers" copy paster? That's the issue being discussed.
20 years and many new technologies of difference.
What is the functional difference between copying an AI answer and copying a StackOverflow answer, in terms of it being "cheating" during an interview?
I think the entire question is missing the forest for the trees. I have never asked a candidate to write code in any fashion during an interview. I talk to them. I ask them how they would solve problems, chase down bugs, or implement new features. I ask about concepts like OOP. I ask about what they've worked on previously, what they found interesting, what they found frustrating, etc.
Languages are largely teachable, it's just syntax and keywords. What I can't teach people is how to think like programmers need to: how to break down big, hard problems into smaller problems and implement solutions. If you know that, I can teach you fucking Swift, it isn't THAT complicated and there's about 5 million examples of "how do I $X" available all over the Internet.
> Languages are largely teachable, it's just syntax and keywords.
This is like "learning a natural language is just 'cramming vocabulary and grammar' - voila, you've become a fluent C1 speaker". :-)
Seriously: if you argue this way, you have only seen a very biased set of programming languages, and additionally, your knowledge of these programming languages is very superficial (i.e. you have never gotten to the "interesting"/"deep" concepts that make this particular programming language special, and which are hard to replicate in most other programming languages).
I think the argument is that for a good chunk of business work, you don't need to use the "interesting"/"deep" concepts. Sure, you'll need time to adapt to the idioms of the language you're using, but following examples you can be just as productive as others in a relatively short time.
> I think the argument is that for a good chunk of business work, you don't need to use the "interesting"/"deep" concepts.
That's what the MBA people want to believe. To lower costs, or if they see writing code as an operating expense, instead of R&D.
If this is true or not, it depends on many, many factors, and it can change over the course of the business life.
> but following examples you can be just as productive as others in a relatively short time.
This is not something nice to say about the colleagues. :-)
But companies don't pay high salaries for the 80% mundane and easy tasks you do day to day. They pay for the 20% that is challenging and especially for that 1% of problems that occur only once every few months or years. If that 80% was 100% of the job then the company could pay 1/2 to 1/3rd the amount by outsourcing it.
I disagree, companies pay based on the problems you can solve to make them money or help achieve organizational goals. One of those ways can be coding, but there are many others.
In C, implicit type narrowing/widening behavior stumped me as a noob working on noob problems. “Deep C Secrets” was a revelation when I finally found it.
Then again, that’s C.
No, the comparison to natural languages is what is whack. If you understand the underlying concepts that programming languages pick and choose from as features, all you have to learn is what keywords map to those concepts and the language's syntax.
The comparison to natural languages would be if you could learn one language and then quickly pick up every other language of that "class" after learning how that single language works. That's not really how natural language works at all, but it does work with programming languages.
> If you understand the underlying concepts that programming languages pick and choose from as features, all you have to learn is what keywords map to those concepts and the language's syntax.
If you understand the grammatical topics that a natural language picks, all you have to learn is what word transformation rules map to those concepts, and the natural language's vocabulary.
> The comparison to natural languages would be if you could learn one language and then quickly pick up every other language of that "class" after learning how that single language works.
There do exist books on this topic (though more commonly for language families). See for example
https://www.quadrilingual.com/
or the book
EuRom 5. Leggere e capire 5 lingue romanze
> That's not really how natural language works at all, but it does work with programming languages.
... it might give you some shallow knowledge in a very limited subset of programming languages.
> If you understand the grammatical topics that a natural language picks, all you have to learn is what word transformation rules map to those concepts, and the natural language's vocabulary.
Yes, and then do that in real time while you're having a conversation with someone who's been learning the language since they were a baby. It is an unreasonable comparison.
> Languages are largely teachable, it's just syntax and keywords.
That's only true for a subset of programming languages, and it requires you to already know how to program in at least another language of the same family. Knowing Java will not help you with Haskell, but it will help you with C#.
I have to deal with students using AI to cheat on homework and exams, and I can't allow them to not even learn the basic concepts.
They could convince you with buzzwords, get hired, and then feed all problems to the AI until it reaches a point where the codebase is too big for the context, and then all their prompt “engineering” experience is basically useless.
That is the future I am trying to prevent.
Until the AI can code a full system like SAP, or an Operating System, or a Photoshop clone, by itself, we need some people in the loop, and the more knowledgeable the people, the better.
> That's only true for a subset of programming languages
That's true, but most of the industry is running on a subset of programming languages.
> That's only true for a subset of programming languages, and it requires you to already know how to program in at least another language of the same family. Knowing Java will not help you with Haskell, but it will help you with C#.
In this context, to a large extent it holds. Yeah. It’s probably more true of mainstream languages roughly related to each other, but in an interview, you’re not testing for knowledge of syntax and keywords. You’re trying to test for ability to build and problem solve.
I share your concern about prompt “engineers” who don’t understand the underlying codebase, language or system. But I don’t know what to do about it in the context of a technical interview.
I'm not arguing to let people cheat with AI. I'm saying asking people to write code in interviews is useless.
It's not useless because some people will lie and cheat. Over the years, I've interviewed hundreds of people and a substantial minority could not write even the simplest code. Many will say they would be able to filter out such people after a conversation. But, IMO, the fact that they are still able to get hired at some places shows how wrong that often is.
I am making the parallel because I feel exams and interviews share some similarities.
I've accidentally been using an AI-proof hiring technique for about 20 years: ask a junior developer to bring code with them and ask them to explain it verbally. You can then talk about what they would change, how they would change it, what they would do differently, if they've used patterns (on purpose or by accident) what the benefits/drawbacks are etc. If they're a senior dev, we give them - on the day - a small but humorously-nasty chunk of code and ask them to reason through it live.
Works really well and it mimics the what we find is the most important bit about coding.
I don't mind if they use AI to shortcut the boring stuff in the day-to-day, as long as they can think critically about the result.
Yep. I've also been using an AI-proof interview for years. We have a normal conversation, they talk about their work, and I do a short round of well-tested technical questions (there's no trivia, let's just talk about some concepts you probably encounter fairly regularly given your areas of expertise).
You can tell who's trying to use AI live. They're clearly reading, and they don't understand the content of their answers, and they never say "I don't know." So if you ask a followup or even "are you sure" they start to panic. It's really obvious.
Maybe this is only a real problem for the teams that offloading their interviewing skills onto some leetcode nonsense...
This is a fine way. I’ll say that the difference between a senior and a principal is that the senior might snicker but the principal knows that there’s a chance the code was written by a founder.
And if the Principal is good, they should stand up and say exactly why the code is bad. If there's a reason to laugh because it is cliche bad, they should say so.
If someone gave me code with
if (x = 7) { ... } as part of a C eval.
Yeah, you'll get a sarcastic response back because I know it is testing code.
What I think people ignore is that personality matters. Especially at the higher levels. If you are a Principal SWE you have to be able to stand up to a CEO and say "No, sir. I think you are wrong. This is why." In a diplomatic way. Or sometimes. Less than diplomatic, depending on the CEO.
One manager that hired me was trying to figure me out. So he said (and I think he was honest at the time). "You got the job as long an you aren't an axe murderer."
To which I replied deadpan: "I hope I hid the axe well." (To be clear to all reading, I have never killed someone, nevermind with an axe! Hi FBI, NSA, CIA and pals!)
Got the job, and we got along great, I operated as his right hand.
> Using apps like GitHub Co-pilot and Cursor to auto-complete code requires very little skill in hands-on coding.
this is a crazy take in the context of coding interviews. first, because it's quite obvious if someone is blindly copy and pasting from cursor, for example, and figuring out what to do is a significant portion of the battle, if you can get cursor to solve a complex problem, elegantly, and in one try, the likelihood that you're actually a good engineer is quite high.
if you're solving a tightly scoped and precise problem, like most coding interviews, the challenge largely lies in identifying the right solution and debugging when it's not right. if you're conducting an interview, you're also likely asking someone to walk through their solution, so it's obvious if they don't understand what they're doing.
cursor and copilot don't solve for that, they make it much easier to write code quickly, once you know what you're doing.
Nowadays I am on the other part of the fence, I am the interviewer. We are not a FAANG, so we just use a SANE interview process. Single interview, we ask the candidate about his CV and what his expectations are, what are his competences and we ask him to show us some code he has written. That's all. The process is fast and extremely effective. You can discriminate week candidates in minutes.
That process might work for your company precisely because you are not FAANG. You don't get hundreds of applicants that are dying to get in, so people don't have that strong of a motivation to do anything it takes (including lying) to get the job.
I’ve worked at a company with 150,000 employees. The interview process was pretty much as described here. There is absolutely no reason a Big Co needs to operate any differently.
Do you reevaluate them in predetermined intervals to see how your initial expectation matches the outcome?
With each Sprint, presumably.
>we ask him to show us some code he has written
How do you expect them to get access to the property internal Git repo codebase and approval from their employer's lawyers to show it to third parties during the interview?
Sounds like you're only selecting Foss devs and nothing more.
Most people have still written code for school or a hobby project. Maybe I'm missing empathy, but I cannot understand how some developers have no code to show.
If that's the case however, just let them make a small project over the weekend and then do another interview where you ask stuff about what they've made. It's not that deep
> Most people have still written code for school or a hobby project. Maybe I'm missing empathy, but I cannot understand how some developers have no code to show.
First: they might have private code, but not necessarily code to show (I, for example, am rather not willing to show quite some of the code that I wrote privately).
Second: the kind of "code" that I tend to write privately (and into which I invest quite a lot of time) is really different from what I do at work, and what is actually considered "code" by many. It's more like (very incomplete) drawings and TeX notes about observations and proofs of properties and symmetries between some algorithms. Once finished, they will be very easy to systematically transform into a program in a computer language.
This is about very novel stuff, which to explain would take quite a lot of time.
The objective in an interview like this shouldn't be to grade the quality of the code you bring in any sort of scale, but to have a discussion about the options you took. In that sense, it really matters very little what you present as long as we can do a small back-and-forth that lets me into what sort of person you are.
> but to have a discussion about the options you took
I can clearly state that this is not I commonly think about code that I write privately (and also for code that I write for the job only if I must). For private code, I rather commonly start with a "gut feeling" about some unexpected symmetry that the problem that I am working on likely has, then try to formulate these "gut feelings" as mathematical properties, and later theorems. At the end, everything "fits (for outsiders: unexpectly) together".
Thus, there is hardly ever a "option that I took", but rather a "I let everything flow: from the source [my gut feeling] to the sea [which is - ironically - the source (code)]".
> Most people have still written code for school or a hobby project
School was years and years ago, and has nothing to do with my current skills.
From the people i personally know, most do _not_ have a hobby project, even fewer have hobby projects that showcases their technical skills. Nor should they be expected to. Most people have non-programming hobbies.
> I cannot understand how some developers have no code to show.
It's really not that deep, I'm worried if you really cannot understand. I don't code outside of work, I'm not interested in doing it. I'm good at software engineering, not passionate about it. I have a bunch of other hobbies. There's no reason I'd have any code to show now or at any point in the future.
> let them make a small project over the weekend and then do another interview where you ask stuff about what they've made
If I'm paid for it, sure why not I could do that. I won't love it but hey I'm looking for a job, I'll put the legwork in. But if this is the only or the "preferred" interview process for a company, I need to point out that it is deeply discriminatory as it advantages people who have the time to do a weekend project: for example it benefits males disproportionally (women do most of the care work in any country, also the most house work, also have a higher chance to be a single parent, all of which impacts the time they can put in a "weekend project" if they can do it at all).
> It's really not that deep, I'm worried if you really cannot understand. I don't code outside of work, I'm not interested in doing it. I'm good at software engineering, not passionate about it. I have a bunch of other hobbies. There's no reason I'd have any code to show now or at any point in the future.
It's like asking a dentist interview candidate to show you examples of fillings and crowns they did at home as a hobby. I don't understand why there is this automatic assumption that people who program at work also do it outside of work.
I ask for code, and if they have none prepared I ask that they spend at most two hours building something they enjoy.
I expect a decent developer to be able to bootstrap and write most of a fun toy project in a domain they know well or at worst some kata from the Internet within half an hour. Then we spend some time screen sharing and talking about it, similar to pair programming but less problem focused.
If you can't do it you'll likely struggle a lot when working with us because we commonly use throwaway prototypes.
Which is fine, you don't have to take in every candidate, nor do we need to apply to every company.
Sure. A bigger organisation with more layers in their technology department could treat candidates better that would struggle with a hiring process like mine. More detailed tickets, more mentors available, bigger cashflow, things like that.
In such settings it's also somewhat common to hire consultants in bulk, like 5-10 at a time, try them out for six months and keep the ones that enjoy the work and fit well in the organisation, and over time try to employ some of them directly.
[flagged]
The equipment is irrelevant. What other career out there expects that practitioners also perform the work at home during their free time as a hobby?
I realize it's a rhetorical question, but pretty much everybody in the arts? I'd be very surprised to find a professional musician that doesn't play as a hobby.
Artists and musicians surely have portfolios to show, but I don't think those are usually composed of things they're doing in their free time. Maybe? I'm not an artist so maybe I'm wrong. I guess then we're back to "you should have a portfolio" rather than specifically "you should do your job in your free time." I can agree with that!
I have written software for laboratory devices on similar price levels, that will never run on home computer.
If you have literally never written a line of code outside of your work in the last decade, you are not a culture fit. This itself is a filter.
If you're not interested, you're not interested. Not even about "passion" at that point, but the bare minimum interest in your industry. I said it better previously: https://news.ycombinator.com/item?id=15553482
I don't code much outside of work. I have hobby projects from 10+ years ago, but they're not much more than landing pages copied from templates and wordpress installs. I mostly work in backend/data/platform engineering professionally.
If I were asked to make a small project over a weekend, I'd be likely to decline rather than doing a more standard interview, or I'd use AI to do it in a reasonable timeframe (which seems to defeat the purpose as it relates to this discussion)
This points to another issue - when I do code outside of work, it's often specifically to try out things I don't do at work. After a day of doing backend work, I'll maybe put together a basic web UI for something. That code is likely awful because I just need it to be functional more than good, and also probably not related to the work I'd be hired to do.
My most recent real "side projects" are a terrible OSS monte carlo simulator tool that I contributed to, but cannot explain most of the code for, and a half-working React application that has performance issues I never fixed. Both are years old at this point. I'm not sure what an interviewer would gain from those.
This relates to another issue of using people's public github as a hiring signal. I don't share any of these repos because at a glance the code is ugly, broken, incomplete.
Below the surface, I'm probably scratching a very interesting itch. Exploring a specific idea or problem, and then I stop when I get my answer.
I started writing code when I was 12 and started doing it professionally at 22. I'm now in my mid-30s and outside of work, I haven't written anything more than one-off scripts for my homelab in close to a decade. I'm already spending upwards of 50 hours with code each week and I need to do something else at night and on the weekends to release my brain from it. I also didn't go to school for CS, and even if I did... it was over a decade ago. So I have ~25 years of experience writing code but could not show you a single line of it. And even if I could, how would you know I was the one to write it?
This is an extremely flawed interview process in my opinion and the last time I encountered it led to an awkward scenario that led to me walking out. Personally, when I conduct interviews, it's a mix of things. We talk about your past work, I quiz you a bit on some topics you'd encounter in your day-to-day here, and then we'll spend an hour doing some combination of a code review of a working-but-flawed demo project I created, a 30-40 minute coding exercise, and/or a problem-solving scenario where I give you a problem and then we talk through how, as a pair, how we could solve it.
School was a few decades ago, and the code I have on Github is mostly toy stuff I do in rainy weekends, most of us have a life without room to code outside work most of the time.
Friends, family, stuff to take care of.
So? That's all that's on our Githubs, too. Show us, talk about it. Let us see we're the same.
I am lucky, that closing 50 years old, still get to code out of work.
Not everyone gets to do so.
Like many of the other commenters, I have no code to show. I'm strongly motivated at work to solve problems and create correct, performant, maintainable code. I appreciate a job well done.
Outside of work, I just don't have the motivation to code anything. I don't have sufficient at-home problems where code will fix them.
In an interview, ask me anything! ... except to show you code on Github.
>Maybe I'm missing empathy,
Worse actually. There is more to life than code - unless you are a savant. Most of us aren't.
But it is the way you are, you probably know no better and you are doing your best, what you can do is to refuse to interview.
Please share your GitHub @
I’ve been working professionally for almost 30 years. I have never written a single line of code “for fun”. I write code for money. I then take that money to fund my hobbies. The absolutely last thing I want to do when I get off work is stare at a computer.
If I already have a job, unless you are paying top of market, why would I spend my weekend writing code?
It looks like you professionally sold 30 years of your life for money with no fun. You could have done something for fun all this time, and got payed for it, too. Much better that way.
I did no such thing, I spent the first 15 years as a part time fitness instructor mostly for the social aspect, hanging out with friends and training for and doing group runs and a little travel.
I spent the next 8 years married (still married) and raising two step sons and spent the last two and half years traveling extensively including over a year doing the “digital nomad thing”.
We have been averaging getting on a plane to do something on average over a dozen times a year since late 2021.
Of course the Covid lockdown slowed us down for two years.
When I am at home in Florida, I go swimming at one of the multiple pools or workout at one of the two gyms that’s part of our complex. It’s warm enough most of the year at least during the day.
During the weekends, I go downstairs and hang out at the bar and just sip soda while hanging out with my friend the bartender and whoever else is down there.
I “retired my wife” in 2020 when I was 46 and she was 44 8 years into my marriage so she could pursue her passion projects and we could pick up and travel as often as we wanted to - the joys of working remotely.
Great for you. You are almost certainly not the type of candidate these companies are looking for.
School? You want to see my 1999 Java code? I’ll go dig out the 3.5 floppy for you.
I’ve written code for hobby projects. It’s mostly HTML, JavaScript and Bash.
I’m a data engineer, so at work I mostly use SQL, Python and Bash. There’s not much overlap.
Who are these "most people"? School was over 10 years ago for me when schoolwork was not posted on GitHub nor is it relevant to my current job anymore, and I don't do hobby coding since I have other hobbies and responsibilities.
WTF is this hobby coding bullshit expectations? What other professions expect you do more work after work as a hobby and show it? Do bus drivers film themselves driving busses after work as a hobby? Do surgeons cut up people in their spare time as a hobby?
The amount of time people spend grinding leetcode instead (if you believe what they say about it online) is just as much or more.
Those are bubbles. I am aware such people exist but nobody I know grinds leetcode or does hobby programing.
It's that trimodal-comp thing. The 6-hour leetcode gauntlet is only the overwhelming norm in the top hump of that graph. It's not normal in the other two (not that it's never seen, but the companies dumb enough to do it without offering FAANG-tier comp are shooting themselves in the foot). I've had, IDK, ten plus tech jobs and have been solidly in the middle comp tier for most of those, and have exactly once encountered anything resembling a leetcode question in the wild.
I've also done hiring, and have no idea what good leetcode would do me. I'm convinced these people thinking they "dodged a bullet" on "fakers" either spotted a real faker who they would have caught with a normal conversation anyway, or else are assuming someone with a good resume who failed their test did so because they were lying and not because they choked under the unique sort of pressure interviews present when you turn them into a dancing-monkey routine (it's approximately the same kind of stress as doing an open mic night or karaoke in front of a crowd of strangers—most people have trouble with that and lots of them fall apart at least some of the times they try it). Meanwhile, anyone who can talk about the job and their career in any depth, convincingly, without giving away that they're actually either very-green or have no idea what they're doing, while in fact not knowing how to do the job, possesses a skill at least as valuable as programming, and I find it hard to believe most such folks haven't figured out they can apply that skill directly in exchange for money instead of trying to fake their way through conversational tech interviews.
I do see how leetcode is valuable if you want to ensure that most of the candidates in your pipeline would do fine before you even evaluate them, because you offer high comp and need a way to discourage candidates who definitely can't make it before they even apply, and/or if you want to make job-hopping painful as a wink-and-nudge collusion way to keep comp suppressed. It makes sense for FAANG, in a certain way, but not because it's a good way to evaluate candidates per se.
Are the companies that want you to do hobby projects and take home test paying close to FAANg level or closer to enterprise Dev level comp?
> WTF is this hobby coding bullshit expectations? What other professions expect you do more work after work as a hobby and show it? Do bus drivers film themselves driving busses after work as a hobby? Do surgeons cut up people in their spare time as a hobby?
I think programming has more commonality with other creative, 'soft' jobs like graphic design (which itself can involve programming), architecture, media, marketing, etc than meets the eye.
Many of these roles require that applicants have some sort of portfolio that can be perused by the interviewer freely. I feel co-opting that word—'portfolio'—would do us software developers a big favour instead of trivialising outside-of-work programming as 'side projects' or 'hobbies'.
I got to see where architects build bridges over weekends as an hobby.
>architecture, media, marketing, etc than meets the eye.
I disagree. Programing is more engineering than art. Art doesn't have source code. You can show the final painting and I can show the final product I worked on but not the source code I wrote as that belongs to my employer. Also, most art like paintings are not done by large teams, so you can show what you did in that painting but in a large SW projects, I can't show what exactly form the final product I did and what else was done by my team.
Most of my valuable work in programing is engineering, especially fixing bugs, not creating portfolios to show off. I have nothing publicly to show off, mostly because firstly, it's private to my former employers, and secondly because code gets outdated and replaced fast, most of what I worte in the past probably doesn't run today anymore, but have made my employers happy and wealthy.
Sounds like you just treat programming more like engineering than art. Some art does have source code, there is plenty of room for creative exploration with code.
>there is plenty of room for creative exploration with code.
That also pays the bills? That's not my experience. That's what hobbies are for. Jobs are for paying bills. Paying bills with hobbies an art are a luxury for privileged.
I said neither 'art' nor 'paintings' which you have fixated on for some reason. I mentioned creative endeavours that are generally team-based but all generate some sort of portfolio. Whether that portfolio is from work or done in one's personal time, it is still a portfolio of past work.
Plus, software engineering is absolutely a creative endeavour. And I daresay normal 'engineering' (civil, mechanical, aero, etc) is a creative endeavour too; it's just a matter of egos and that seem to separate STEM versus non-STEM. There are portfolios for everything. I don't understand the desire for software engineers to just waltz into an interview, claim to have done X, Y, Z, with no proof, and secure a job.
The proof part is interesting. Civil is easy to prove because of its artifacts. Someone from Netflix or Meta layoffs, what proof do any of them have? Do some people defensively maintain background proof other than paycheck stubs?
> Art doesn't have source code.
Drawn art absolutely does have something like it:
https://www.reddit.com/r/learntodraw/comments/nibjjn/any_adv...
It could be considered similar to scaffolding or boilerplate in code, except usually none of this is visible in the end product, while the code boilerplate is always there. These lines are drawn light and completely covered up by the end result - sometimes even manually erased depending on the medium.
I don't like writing code for the sake of it, and have gotten a lot better, over 25 years of writing code, at evaluating whether I need to write code or whether I'd be better off using something that already exists and putting up with its limitations, or even just doing nothing (see that XKCD comic with the time-savings payoff chart).
The result is that I don't think I've written anything longer than about a ten-line shell or python or JS script for my personal use in... a decade or more.
Frankly I probably think you shouldn't be paying anyone to do the thing you're wanting to pay me to do, because computers are likely just an expensive distraction that management's pursuing because the promise of legibility, even if in-fact pointless in this case, is incredibly enticing to them, but also I like money and will build the thing you shouldn't be building for you if you pay me. I'll even do it well, if you let me. But I don't make the same mistake (much) in my own life, any more.
Would I write a bunch of code on my own if I thought it'd be worth it? Yes, but that'd almost certainly mean I had a product idea. If I were any good at thinking of product ideas, I'd long since have had my own business. I'm terrible at it. That's literally the only reason I'm applying for a job. If I had a pile of decent code to show you, it'd be because I didn't need your job.
> WTF is this hobby coding bullshit expectations?
I can empathize with your position if you are also against expecting candidates "prepare for the interview" by leet code grinding or "brush up on CS concepts".
Hobby coding is million times better than that crap.
At least leetcode is something somewhat standardized (algorithms don't change). Hobby coding is not, it's something subjective and varying between the interests of each candidates and often have nothing to do with a job.
My worst code is always what I wrote yesterday. Often what’s missing is context, unless I comment ad nauseam. Sure I didn’t write complete test, obey open closed principles abstract into factory functions. The code I send from my hobby projects is likely a mess, because finishing on my own time by my own unpaid constraints wills it to be so
Maybe you forked a library because of reasons. You can tour the original repo and explain the problems. I have at least one of those examples for each time the legal or confidentiality department stepped in.
>Maybe
The word maybe is doing a lot of heavy lifting here. What if you never had to do that? Not everyone's work is public. Inf act I'd say most people's work is not public. Sometimes even the product is not public since it's B-2-B.
But if you’ve worked on something mature and nontrivial, you’ve forked a dependency and are able to tour it. Looks like I’ve done it on average twice per year.
I’ve worked on a piece of software you have at least heard of, probably used, and it is so far from publicly available it’s funny.
I never had to for anything public.
*weak
We do this too, works fine. We ask open ended questions like, "What's your favorite thing you've done in your career and why?" and "What was the most challenging project in your career and why?" If you listen, you can get a lot of insight from just those two questions. If they don't give enough detail, we'll probe a little.
Our "gotcha," which doesn't apply to most languages anymore is, "What's the difference between a function and a procedure." It's a one sentence answer, but people who didn't know it would give some pretty enlightening answers.
Edit: From the replies I can see people are a little defensive about not knowing it. Not knowing it is ok because it was a question I asked people 20 years ago relevant to a language long dead in the US. I blame the defensiveness on how FUBAR the current landscape is. Giving a nuanced answer to show your depth of knowledge is actually preferred. A once sentence answer is minimal.
I'm editing this because HN says I'm posting too fast, which is super annoying, but what can I do?
> We do this too, works fine. We ask open ended questions like, "What's your favorite thing you've done in your career and why?" and "What was the most challenging project in your career and why?" If you listen, you can get a lot of insight from just those two questions. If they don't give enough detail, we'll probe a little.
The problem is: there is a very negative incentive to give honest answers. If I were to answer these questions honestly, I'd bring up some very interesting theorems (related to some deep algorithmic topics) that I proved in my PhD thesis. Yes, I would have loved to stay in academia, but I switched to industry because of the bad job prospects in academia - this is not what interviewers want to hear. :-(
> "What's the difference between a function and a procedure." It's a one sentence answer
The terminology here differs quite a lot in different "programming communities". For example
> https://en.wikipedia.org/w/index.php?title=Procedure&oldid=1...
says: "Procedure (computer science), also termed a subroutine, function, or subprogram",
i.e. there is no difference. On the other hand, Pascal programmers strongly distinguish between functions and procedures; here functions return a value, but procedures don't. Programmers who are more attracted to type theory (think Haskell) would rather consider "procedures" to be functions returning a unit type. If you rather come from a database programming background, (stored) procedures vs functions are quite different concepts.
I could go on and on. What I want to point out is that this topic is much more subtle than a "one sentence answer".
> I'd bring up some very interesting theorems (related to some deep algorithmic topics) that I proved in my PhD thesis. [...] I switched to industry because of the bad job prospects in academia - this is not what interviewers want to hear.
In my experience you'll be fine giving that answer assuming you're going for the kind of programming job that hires PhDs.
You remind them you have a PhD - and in something deeply algorithmic. You can successfully answer any follow-up questions from them, as you literally have a PhD in the topic they're asking about. There's no shame in entering industry because you want jobs and money - in fact, those things are precisely what the hiring manager is able to offer you.
You'd rather be in academia but it doesn't have the pay and job security? Well, the hiring manager would rather be a snowboard instructor in Aspen but doesn't for the same reason. So you've got common ground with them.
> Yes, I would have loved to stay in academia, but I switched to industry because of the bad job prospects in academia - this is not what interviewers want to hear. :-(
I would love to hear that from a candidate I'm interviewing. Who can't relate to the distinction between your ideal job and the job that will actually pay you money?
>The problem is: there is a very negative incentive to give honest answers. If I were to answer these questions honestly, I'd bring up some very interesting theorems (related to some deep algorithmic topics) that I proved in my PhD thesis.
This is unfortunate that you would get that response. FWIW, I would be interested in hearing all this in an interview and I would look at it favorably.
>What I want to point out is that this topic is much more subtle than a "one sentence answer".
Yes, you would definitely get bonus points for nuance. The one sentence answer was minimal. What it filters out are people who don't know anything about Delphi but applying for the job with highly embellished resumes hoping to get lucky. This was for software used in hospitals, so bugs or errant code could have pretty drastic consequences.
Here's an interesting thought on your "gotcha" - I'm 57 years old, been programming as a career for over 30 years, a lot of languages and I have no idea what the difference is.
If I'm applying for a Java position and I claim to have Java experience on my resume, it's perfectly valid for them to ask me the difference between an int, an Integer, and a BigInteger.
But it's certainly not a universal question applicable to all programming languages.
Likewise, Clubber says in their post that their 'gotcha' question doesn't apply to most languages.
I have no idea either. I can easily look it up though. You can often tell an inexperienced interviewer from the extremely domain specific question they ask which _they_ are familiar with.
There are some absolutely ridiculous qs I've been asked like this and they've all had no followup question to illuminate why it would have been relevant 1. what version of java do you use? we used 8 at the time 2. what is the engine and version underneath your sql db?this was not for a dba role, just standard backend engineer 3. why did you use python instead of r for x project? this was about a gui automation script
>You can often tell an inexperienced interviewer from the extremely domain specific question they ask which _they_ are familiar with.
Lol a bit touchy aren't we?
Like I said, it's not really relevant in today's languages. It was for a Delphi/Pascal position. If you do any type of database code (like T-SQL), you would also know it. If your experience is mainly in C type languages, everything is a function so it doesn't apply.
If you hired a guy for a Delphi position who didn't know the difference between a function and a procedure, you hired the wrong guy.
Function or procedure is defined in every subroutine. It's a very basic question for Delphi, like what's the difference between an integer and a string.> Our "gotcha," which doesn't apply to most languages anymore is, "What's the difference between a function and a procedure." It's a one sentence answer, but people who didn't know it would give some pretty enlightening answers.
If you're asking this question (by virtue of the present-tense "is") in the year 2025 even though by your own admission
> it's not really relevant in today's languages.
then you aren't giving candidates a good impression. Even though I would have nailed this question I would have serious reservations about any job that would ask it in an interview because it means that the person interviewing me has more concern for legacy minutiae than broad technical knowledge or problem-solving skills.
Ya sorry, this was way back in the late 90s-early 00s. It's still only relevant in SQL dev.
>Lol a bit touchy aren't we?
No. I just looked up other responses to your post. It's obvious you got exposed as being inexperienced (or an idiot), while posing to know the definitive with your "gotcha". Being inexperienced (or ignorant) is not a problem, but being cocky is.
I think you're projecting. You aren't posting this using your real name are you?!
Here's something to consider, Dennis. Instead of using any type of reasoning that maybe I'm interviewing for a language you aren't familiar with where functions and procedure differences matter, you decided to just go off the handle and call me inexperienced and/or an idiot. This is what we call in the hiring business a "huge red flag." I recommend maybe use some of that big brain you have and apply some deductive reasoning instead of just calling people names.
Look up my other responses - I decided to call you names _later_ . Other people also pointed that out to you. I'm sure you would thick twice before asking your 'gotcha' again (the question is fine in general but not as a gotcha).
>>You can often tell an inexperienced interviewer from the extremely domain specific question they ask which _they_ are familiar with.
>Lol a bit touchy aren't we?
You lost your composure and decided to start calling names after this. I haven't asked it since the late 90s, early 2000s. It was a for a Delphi position. It's a bonehead easy question any Delphi developer who got to chapter 2 of any Delphi book would have understood. It's still an applicable question for a SQL developer and it's just as easy. I even showed you sample code. I don't see why you aren't getting it.
I'm not getting it, I also see that 2 other people are also not getting it.
>I haven't asked it since the late 90s, early 2000s.
You overall gave the impression that you are currently asking it.
And a personal rhetorical question - aren't you too old to even state this 'gotcha' business _today_ about what you did in the past? What made you state it? If I gave you the benefit of doubt - that was slip, where you omitted the past tense.
(If I did that in the 90's I'd be a embarrassed to even mention it today.)
>I'm not getting it, I also see that 2 other people are also not getting it.
Yes, but you're the only one who threw a tantrum. Enjoy your career.
I think you are missing my point.
Words like functions/procedures tend to have different connotations across languages and once one crosses one's 15th language, and each having some 20 different keywords, it become difficult to remember what the exact connotation of a word is, in a specific language/framework. This is the most likely situation of the guy whose post I responded to.
The exception to the rule is, if you have been working quite a bit _recently_ on a specific language. You are presumably talking about this situation.
It’s ok to say that it’s never professionally mattered. No one has ever been paid to know that. “Are side effects a bad pattern?” Lotsa people have needed to know that on day one.
A function returns a value.
https://stackoverflow.com/questions/721090/what-is-the-diffe...
> A function returns a value
... in Pascal/Delphi.
Yup, it was for a Delphi job in 1998.
Isn't this something the compiler is going to enforce anyway? Why does it matter? If that's the only difference it's kind of inconsequential isn't it?
It's a question to filter out people who don't know a lick of Delphi for a position where the person was coding Delphi. It's like level 2 after the hello world chapter. It's easier than asking a database developer the difference between a query and a view. It's a bonehead simple question that 70% of applicants couldn't answer.
You would be surprised how many bad applicants interviewers get. It's only gotten worse over the last 20 years.
> Our "gotcha," which doesn't apply to most languages anymore is, "What's the difference between a function and a procedure."
My answer would be along the lines of "It's 2025, no one has talked about procedures for 20+ years"
> Single interview, we ask the candidate about *his* CV and what *his* expectations are, what are *his* competences and we ask *him* to show us some code *he* has written
You... might want to think about what implicit biases you might be bringing here
What I've been thinking about leetcode medium/hard as a 30-45 minute tech interview (as there are a few minutes of pleasantry and 10 minutes reserved for questions), is that you are only really likely to reveal 2 camps of people—taking in good faith that they are not "cheating". One who is approaching the problem from first principles and the other who knows the solution already.
Take maximum subarray problem, which can be optimally solved with Kadane's algorithm. If you don't know that, you are looking at the problem as Professor Kadane once did. I can't say for sure, but I suspect it took him longer than 30-45 minutes to come up with his solution, and I also imagine he didn't spend the whole time blabbering about his thought process.
I often see comments like: this person had this huge storied resume but couldn't code their way out of a paper bag. Now having been that engineer stuck in a paper bag a few times, I think this is a very narrow way to view others.
I don't know the optimal way to interview engineers. I do know the style of interview that I prefer and excel at[0], but I wouldn't be so naive to think that the style that works for me would work for all. Often I chuckle about an anecdote from the fabled I.P. Sharp: Ian Sharp would set a light meter on his desk and measure how wide an interviewees eyes would get when he explained to them about APL. A strange way to interview, but is it any less strange than interviewing people via leetcode problems?
0: I think my ideal tech screen interview question is one that 1) has test cases 2) the test cases gradually ramp up in complexity 3) the complexity isn't revealed all at once; the interviewer "hides their cards," so to speak 4) is focused on a data structure rather than an algorithm such that the algorithm falls out naturally rather than serves as the focus. 5) Gives the opportunity for the candidate to weigh tradeoffs, make compromises, and cut corners given the time frame. 6) Doesn't combine big ideas (i.e. you shouldn't have to parse complex input and do something complicated with it); pick a single focus. Interviews I have participated and enjoyed like this: construct a Set class (union, difference, etc); implement an rpn calculator (ramp up the complexity by introducing multiple arities); create a range function that works like the python range function (for junior engineers, this one involves a function with different behavior based on arity).
>Take maximum subarray problem, which can be optimally solved with Kadane's algorithm. If you don't know that, you are looking at the problem as Professor Kadane once did. I can't say for sure, but I suspect it took him longer than 30-45 minutes to come up with his solution, and I also imagine he didn't spend the whole time blabbering about his thought process.
This is something that drives me nuts in academia when it comes to exam questions. I once took an exam that asked us to invent vector clocks from whole cloth, basically, having only knowledge of a basic Lamport clock for context. I think one person got it--and that person had just learned about vector clocks in a different class. Given some time, it's possible I could have figured it out. But on an exam, you've got like 10-15 minutes per question.
The funny thing about it is that I do the same damn thing from the other side all the time when working with students. It's incredibly tempting once you know the solution to a problem (especially if you didn't "solve" it yourself, but had the solution presented to you already) to present the question as though it has an obvious solution and expect somebody else to immediately solve it.
I'm aware of the effect, I've experienced it many times, and I still catch myself doing it. I've never interviewed a candidate for a job, but I can only imagine how tempting it would be to fall into that trap.
Yes that's a tricky one.
When I'm interviewing a candidate, I'm often asking myself if this question is just something I happen to know therefor expect the candidate to know too, or if it's crucial to doing the job?
Sometimes it may not be fair to expect a random developer to be familiar with a specific concept. But at the same time it might be critical to the kind of work we're doing.
The current job market is so messed up that I honestly can't see myself getting a job until we hit a wall and people start using their brains again.
I have 26 years of solid experience, been writing code since I was 8.
There should be a ton of companies out there just dying to hire someone with that kind of experience.
But I'm not perfect, no one is; and faking doesn't work very well for me.
> There should be a ton of companies out there just dying to hire someone with that kind of experience.
heh.. they are probably dead already?
i have even longer years.. But this time i am looking since.. september? Applying 1-2 per day, on average.. Widening the fishing net each month.. ~2% showed some interest.. but no bingo.
"overqualified" is about half of the "excuses" :/
Time to plant tomatoes maybe..
Or maybe join forces and show them how it's really done?
Not that I mind growing tomatoes, quite the opposite :)
I am with you! Been programming since I was 10 and have 20YoE. Many of my prototypes have grown into full fledged products, I have 40+ published papers, and I am regularly sought out for advice and help by those who know me. Everyone i have been, I am always told I am a good catch.
However, I won't do leet coding. I want to hear about why I should come work for u. What about my works makes u think I could help ubm with your problem. Then let's have a talk about your problems and where I can create value for you.
My experience in hiring is that leet coders are good one trick ponies. But long term don't become technical peers.
Part of the problem is there just aren't a lot of people out there who can correctly judge that level of experience, and looking up the spectrum tends to simply look weird.
I agree. Do you have any thoughts on how to mitigate this? After all, its in your best advantage and the companies to hire someone with talents because of the value they can bring.
The problem isn't AI, the problem is companies don't know how to properly select between candidates, and they don't apply even the basics of Psychometrics. Do they do item analysis of their custom coding tests? Do they analyse the new hires' performances and relate them to their interview scores? I seriously doubt it.
Also, the best (albeit the most expensive) selection process is simply letting the new person to do the actual work for a few weeks.
[1] https://en.wikipedia.org/wiki/Psychometrics
> Also, the best (albeit the most expensive) selection process is simply letting the new person to do the actual work for a few weeks.
What kind of desperate candidate would agree to that? Also, what do you expect to see from the person in a few weeks? Usual onboarding (company + project) will take like 2-3 months before a person is efficient.
Candidate would be compensated, obviously. That's why it's expensive.
You don't need him to become efficient. Also I don't think it is always necessary to have such long onboarding. I'll never understand why a new hire (at least in senior position) can't start contributing after a week.
> Candidate would be compensated, obviously. That's why it's expensive
Ok... take me through it. I apply to your company and after a short call you offer me to spend 4 weeks working at your place instead of an interview.
I go back to my employer, give them resignation letter, work the rest of my notice period (2 months - 3 months), working on all handovers, saying goodbyes.
Unless the idea is to compensate me for the risk (I guess at least 6 months salary, probably more), then I do not see how you'd get anyone who is just a poor candidate to sign up for this.
> You don't need him to become efficient
So what will you see? Efficiency, being independent and being a good team player are the main things that are difficult to test during a regular interview.
And so that self-selects for people who already are unemployed then, right? Most developers I know (including myself) look for a new job while still having a job, as to not create a financial hole in-between. I'd be curious if that doesn't then end up with lower quality candidates who ended up unemployed to begin with?
And, additionally, it encourages your candidates to still be interviewing while they're on their probationary period with you, since they may be back to unemployed after 4 weeks or whatever. Which creates even more potential issues if they get a much better offer while they're onboarding with you.
> self-selects for people who already are unemployed then
You can say that about all forms of hiring process. If you're unemployed, you obviously have more time: to spend more time on the take-home assignments (which I hate, see another thread [1]), to add more stuff to your GitHub profile, to go to more interviews, etc.
[1] https://news.ycombinator.com/item?id=40200397
> You can say that about all forms of hiring process
Yes, but there's a significant difference between spending a few hours on a take-home assignment and dropping your current employment to spend 4 weeks potentially in another city working full time.
Well, I didn't say it was super practical approach, only that it has the best predictive validity :D
I'd argue the bigger expense is on the team having to onboard what could potentially be a revolving door of temporary hires. Getting a new engineer to the point where they understand how things work and the specific weirdness of the company and its patterns is a pretty big effort at anywhere I've worked.
> can't start contributing after a week.
Because you have zero context of what the org is working on.
If you work with Boring Technology, your onboarding process has no reason to be longer than a week, unless you're trying to make the non-tech parts of the role too interesting.
> unless you're trying to make the non-tech parts of the role too interesting.
Unless your role is trivial to replace with an LLM, you need to understand the business. Maybe not for really junior role, but everything above - you need to solve issues. Tech is just a tool.
You don't have to understand the entire business to start being productive. Particularly when you have experience from other, similar businesses.
I am not sure I follow - when you hire you search for someone who has 100% coverage of the tech you're using and also already works for your direct competitor?
Let's say you're hiring manager for a company that compares flight tickets, something similar to Google Flights or Skyscanner. You need three additional Rust engineers. You're located in Palermo, Italy.
How do you hire people that would not only know Rust, be willing to move to Palermo, or at least visit occasionally, but also know the airfare business?
Even if you're willing to have people remotely, in the same region, how many unemployed Rust developers that know that business are on the market? 0?
> when you hire you search for someone who has 100% coverage of the tech you're using and also already works for your direct competitor?
Ideally, yes. It's a common occurrence among large organisations. Google and Apple used to even have an anti-poaching agreement.
> How do you hire people that would not only know Rust, be willing to move to Palermo, or at least visit occasionally, but also know the airfare business?
Rust isn't Boring, which is why you don't do that and hire one of many Java developers and do Java, unless the tradeoff is really worth it.
How do you control for confounders and small data?
For data size, if you're a medium-ish company, you may only hire a few engineers a year (1000 person company, 5% SWE staff, 20% turnover annually = 10 new engineers hired per year), so the numbers will be small and a correlation will be potentially weak/noisy.
For confounders, a bad manager or atypical context may cause a great engineer to 'perform' poorly and leave early. Human factors are big.
Sure, psychological research is hard because of this, but that's not what I'm proposing - I'm talking about just having some data on predictive validity of the hiring process. If there's some coding test: is it reliable and valid? Aren't some items redundant because they're too easy or too hard? Which items have the best discrimination parameter? How the total scores correlate with e.g. length of the test takers tenures?
Sure, the confidence intervals will be wide, but it doesn't matter, even noisy data are better than no data.
Maybe some companies already do this, but I didn't see it (though my sample is small).
Might as well use https://en.wikipedia.org/wiki/E-meter
I mostly skipped the technical questions in the last few interviews I have conducted. I have a conversation, ask them about their career, about job changes, about hobbies, what they do after work. If you know the subject, skilled people talk a certain way, whether it is IT, construction, sailing.
I do rely on HR having, hopefully, done their job and validated the work history.
I do have one technical question that started out as fun and quirky but has actually shown more value than expected. I call it the desert island cli.
What are your 5 linux cli desert island commands?
Having a hardware background, today, mine are: vi, lsof, netcat, glances, and I am blanking on a fifth. We have been doing a lot of terraform lately
I have had several interesting responses
Manager level candidate with 15+ years hands on experience. He thought it was a dumb question because it would never happen. He became the teams manager a few months after hiring. He was a great manager and we are friends.
Manager level to replace the dumb question manager. His were all Mac terminal eye candy. He did not get the job.
Senior level SRE hire with a software background. He only needed two emacs and a compiler, he could write anything else he needed.
> I have a conversation, ask them about their career, about job changes, about hobbies, what they do after work. If you know the subject, skilled people talk a certain way, whether it is IT, construction, sailing.
My experience differs a lot. Many insanely skilled people are somewhat "weird" (including possibly
- being a little on the spectrum,
- "living a little bit in their own world",
- having opinions on topics that are politically "inappropriate" (not in the sense of "being on the 'wrong' side of a political fence", but rather in the sense of "an opinion that is quite different than what you have ever heard in your own bubble", and is thus not "socially accepted")
- being a little bit "obnoxious" (not in bad sense, but in a sense that might annoy a particular kind of people))
What you consider to be "skilled people" is what I would rather call "skilled self-promoters" (or possibly "smooth talker"). "Skilled people" and "skilled self-promoter" are quite different breeds of people.
> My experience differs a lot. Many insanely skilled people are somewhat "weird" (including possibly
I am actually a bit weird myself, so I can relate.
> What you consider to be "skilled people" is what I would rather call "skilled self-promoters". "Skilled people" and "skilled self-promoter" are quite different breeds of people.
I don't mean that they have told me that they are skilled, or that their resume has implied it. I mean that they actually have the skills. Self-promoters that don't know the information always look good on paper, but after a few minutes of talking to them you can tell that they don't quite match.
Before IT, I was a live sound engineer TV, theater, music. There was also a entertainment university starting up around the same time. They were pumping out tons of "trained" engineers that looked good on paper but couldn't mix for shit. I think we can blame them for the shitification of pop music.
> Self-promoters that don't know the information always look good on paper, but after a few minutes of talking to them you can tell that they don't quite match.
My experience differs here: these are not "good self-promoters", but impostors.
Good self-promoters typically have some above-average (though commonly not really exceptional) skills in their area, but their expertise is in the capability of smooth talking (including smalltalk), promoting their contributions, and talking at eye level with various stakeholders.
If you are really exceptional in your area, you will often (though not always) consider smalltalk to be waste of your time, and will often have difficulties talking at eye level with various stakeholders, because either they are not sufficiently knowledgable in your area of expertise to understand you, or the other way round (for the latter point: becoming really great in one area often means that you won't have the time to get sufficiently deep into a lot of other areas, even though for some of them you might become quite skilled if you had more time).
I have an ice breaker type question which is “what’s something (tool, tech, whatever) you are interested/excited about and wish people knew more about?” Selfishly, interviewing is kind of boring, so I’m always looking to learn something new.
Sadly, out of 100s of people, I’ve probably only gotten an interesting response a handful of times. Mostly people say some well known tech from the job description.
I never held that against anyone, but the people who had an interest in something were more fun to work with.
>I mostly skipped the technical questions in the last few interviews I have conducted. I have a conversation
Sir, you have attained dizzying intellectual heights that few men have.
My comment is meant to be a compliment, not snarky. And indeed I have noticed that the best people I have encountered can often size people up accurately with very general questions often on unrelated subjects.
Thank you. I took it as both. :)
Do you have network access? I would pick ssh.
> blanking on a fifth
grep? (or ripgrep if allowed)
busybox: so many tools bundled into one binary
Excellent answer.
> What are your 5 linux cli desert island commands?
Are you familiar with busybox ?
> Manager level to replace the dumb question manager. His were all Mac terminal eye candy. He did not get the job
Huh? Please explain
We hired the guy who said it was a dump question, he became our manager. He then decided to retire and we had to replace him. One of the candidates answered the 5 cli question with terminal eye candy, not functional commands. He was not hired for the job.
My last interview, for the job I'm currently employed in, asked for a take home assignment where I was allowed to use any tool I'd use regularly including AI. Similar process for a live coding interview iterating on the take home that followed. I personally used it to speed up wirting initial boilerplate and test suites.
I fail to see why this wouldn't be the obvious choice. Do we disallow linters or static analysis on interviews? This is a tool and checking for skill and good practices using it makes all sense.
As someone on the other side of the table, I don't care if you used AI to complete a take-home project. I care if you can explain the strengths and weaknesses of the approach it took, or if you chose to steer it in one direction or another. It usually becomes quite clear those who actually understand what the AI actually did for them.
There’s no other industry* that interviews experienced people the way we do. So maybe just do what everyone else does.
Everyone is so terrified of hiring someone that can’t code, but the most likely bad hires and the most damaging bad hires are bad because of things that have nothing to do with raw coding ability.
*Except the performing arts. The way we interview is pretty close to the way musicians are interviewed, but that’s also really similar to their actual job.
I've been arguing that "AI" has very little impact on meaningful technical interviews, that is ones that don't test for memorization of programming trivia: https://blog.sulami.xyz/posts/llm-interviews/
We have been interviewing people who are obviously using covert AI helper tools. Ask them a question and they respond with coherent response, but they are just reading off of a window we can't see.
In some cases it is obvious they are blathering a stream of words they don't understand. But others are able to hold something resembling a coherent conversation. We also have to allow for the fact that most people we interview aren't native English speakers, and are talking over Teams. It can be very hard to tell if they are cheating.
Asking questions to probe their technical skills is essential, otherwise you are just selecting for people who are good at talking and self promotion. We aren't just asking trivia questions.
We also give a simple code challenge, nothing difficult. If they have a working knowledge of the language, they should be able to work through the problem in 30 minutes, and we let them use an IDE and google for things like regex syntax.
Some of them are obviously using an AI, since they just start typing in a working solution. But in theory they could be a Scala expert who remembers how to use map plus a simple regex...
A couple of weeks ago I interviewed at a place where I had to do a take-home exercise. It's fine, I don't mind. No Leetcode. Just my own IDE, my own shortcuts, and write a piece code that solves a problem.
I was asked whether I used AI/LLM for the solution. I didn't. I felt like using an LLM to solve the problem for me wasn't the right way of showcasing knowledge. The role was for some form of 'come in with knowledge and help us'.
The response to that was basically: everybody here uses AI.
I declined the follow-up interview, as I felt that if all you have is the speed of AI to be ahead of your competitors, you're not really building the kind of things that I want to be a part of. It basically implies that the role is up in the air as soon as the AI gets better.
When I started coding I did it in notepad. I thought it was hardcore and cool. I was young and stupid. Then I adopted an IDE and I became much better at writing code.
To me AI is just another tool that helps me solve problems with code. An auto complete on steroids. A context aware stack overflow search. Not wanting to adopt or not even work somewhere where colleagues use it, sounds to me like coding in notepad AND in the process scoffing those who use an IDE.
Besides, if AI gets to the point it can replace you, it will replace you. Better to start learning how to work with it so you can fill whatever gap AI can't.
I still mainly use a text editor after several decades, and do a lot of thinking and initial design with pencil and paper. IDEs just get in the way.
I've seen the type of code AI generates. It might work, but if you think that's good or that massaging it so it works will make you any better, I have some bad news for you...
That link doesn't work for me.
Prediction: faangs will come up with something clever or random or just fly everyone onsite, they are so rich and popular, they can filter by any arbitrary criteria.
Second-rate companies will keep some superficial coding, but will start to emphasize more of the verbal parts like system design and retrospective. Which sucks, because those are totally subjective and mostly filters for whoever can BS better on the spot and/or cater to the interviewer's mood and biases better.
My favorite still: in-person pair programming for a realistic problem (could be made-up or shortened, but similar to the real ones on the job). Use whatever tools you want, but get the correct requirements and then explain what you just did, and why.
A shorter/easier task is to code review/critique a chunk of code, could even just print it out if in person.
It's not that hard. Just ask them to explain the code. Then ask them how they'd change it for several different scenarios.
I've taken this approach, and found that it's trivially easy to distinguish people relying on LLMs from people who have thought the problem through and can explain their own decision-making process.
I had a couple of people who, when asked to explain specific approaches reflected in their code, very obviously typed my question right back into ChatGPT and then recited its output verbatim. Those interviews came to an end rather quickly.
One of my favorite ones was when I asked a candidate to estimate the complexity of their solution, and ChatGPT got it wrong, giving O(log(n)) for an O(n) algorithm. When I asked leading questions to see if the candidate could see where the error came in, they starting verbatim reciting a dictionary definition of computational complexity, and could not address the specifics of the problem at all.
This whole conversation is depressing me. when I left work a couple years ago due to health reasons, AI was just beginning to become a problem. Now, thanks to a clinical study, I may possibly be able to return to work, and it sounds like the industry has changed drastically.
Not looking forward to it.
It hasn’t. Most businesses have continued operating the way they have for years.
SV Startup hiring is the most trendy and not representative.
I think that the effects at the moment are highly exaggerated in the tech media than the reality on the ground.
How long that will remain true is a very open question, where different folks have widely differing timelines on when they expect AI to have highly meaningful impacts.
Our tech screen is having the candidate walk me through a small project that I created to highlight our stack. They watch my screen and solve a problem to get the app running then they walk me through a registration flow from the client to the server and then returning to the client. There are no gotchas but there are opportunities to improve on the code (left unstated... some candidates will see something that is suboptimal and ask why or suggest some changes).
We get to touch on client and browser issues, graphQL, Postgres, Node, Typescript (and even the various libraries used). It includes basic CRUD functionality, a third party API integration, basic security concerns and more. It's just meant to gauge a minimal level of fluency for people that will be in hands on keyboard roles (juniors up to leads, basically). I don't think anyone has found a way to use AI to help them (yet) but if this is too much for them they will quickly fall flat in the day to day job.
Where we HAVE encountered AI is in the question/answer portion of the process. So far many of those have been painfully obvious but I'm sure others were cagier about it. The one incident that we have had that kind of shook us was when someone used a stand-in to do the screen (he was fantastic, lol) and then when we hired him it took us about a week to realize that this was a team of people using an AI avatar that looked very much like the person we interviewed. They claimed to be in California but were actually in India and were streaming video from a Windows machine to the Mac we had supplied for Teams meetings. In one meeting (as red flags were accumulating) their Windows machine crashed and the outline of the person in Teams was replaced by the old school blue screen of death.
How about paid internships as a way to filter candidates? As in, hire a candidate for a small period of time, like 2 weeks or something, and have them work on a real task with a full-time employee and use their performance on that to decide whether or not to hire.
I realize it's not easy for smaller companies to do, but I think it's the single best way to see if someone's fit for a job
I'm someone who hated leetcode style interviews for the longest, but I'm starting to come around on them. I get that these style of questions are easy to game, but I still think they have _some_ value. The point of these style of questions was supposed to test your ability to problem solve and come up with a good solution given the tools you knew. That being said, I don't think every company should be using this type of question for their interviews. I think leetcode style questions should be reserved for companies that are pushing the boundary of the industry since they're exploring charted territory and need people who can come up with unique solutions to problems no one really knows. I think most companies would be fine with some kind of pairing problem since most people are probably solving engineering problems instead of computer science problems. But none of this matters, since, we all know that even if we went that direction as an industry, the business people would fuck it up some how anyways.
> reserved for companies that are pushing the boundary of the industry
In a world where every company beleives (or wants to beleive) that they are doing some ground-breaking, bleeding edge work (see any tech company blog and you can only find hyped technologies in there), I do not think one can expect companies to do a fair assessment of if they really are doing such work.
I had no idea people took hackerrank as a serious signal rather than as a tool for recent graduates to track interview prep progress. Surely it has all the same issues AI does: you have no way of verifying that the person who takes the interview actually is responsible for that signal.
I don't see AI as a serious threat to the interview process unless your interview process looks a lot like hackerrank.
Your “unless” covers a huge swath of this industry, at the low end and at the high end. Excluding places that do that leaves you with what exactly? Boutique shops filled with 20 year veterans?
What do you mean by the "high" end? I would consider this sort of interview style necessarily precluding such a place from being considered a high-quality work-place. Not only is it a miserable way to interview, it's not an effective signal for engineer quality beyond rapid code snippet production.
> Excluding places that do that leaves you with what exactly? Boutique shops filled with 20 year veterans?
We are on a VC forum—I imagine small shops focused on quality are quite common here.
“High end” was meant as shorthand for FAANG …high comp, not necessarily high tech complexity.
You are correct about the deficiencies of the whiteboard interview. It is not a sane way to hire an individual. It makes sense as a way to hire someone in the top 20% from a large unfiltered pool. So wrt high/low, that’s what FAANG companies have to do, and for many nontechnical companies they outsource this work or emulate FAANG practices for no good reason.
My point was that there are very few places that don’t do this.
Ah, makes sense.
I just abandoned the code interview altogether and ask them questions about process. It's a very simple workaround, but very effective. I'll admit it helps that there are very few problems these days outside of specific problems to solve that require a high degree of technical competency to tackle.
I feel like we, SWEs, have been over-engineering our interview process. Maybe it's time to simplify it, for example, just ask questions based on the candidate's resume instead of coming up with random challenges. I feel like all the new proposals seem overly complicated, and nobody, interviewer or interviewee, is happy with any of them.
Definitely over-engineering. But I also think the industry is just extremely bad at hiring anything above junior or entry-level. Job postings are so generic and interchangeable between companies that they don't actually tell you what the role is or what the company is looking for. Everyone wants to cast the widest possible net so that they catch some wunderkind genius out of thousands. Then, they wonder why they can't find the exact person they're looking for to solve the problem they're filling the role for.
In reality, job postings should be incredibly specific, with specificity rising as the role requires more experience and problem solving. You'll get less applicants (or will be able to clearly screen out the people who don't meet the specific requirements) but you'll get ones that actually match what you are looking for and can actually solve the problem your company is trying to solve with filling the role. Then the conversation/interview is much more important and both sides feel like they have some "stakes in the game".
This risks hiring candidates who can present themselves and their past projects very well but fail to actually write code and ship anything on the job. I've seen it happen.
You would be very amazed at how many people with reasonably strong resumes can't write _any_ code. Google for fizzbuz, it's a dumb problem, but candidates often can't solve similar problems with a _take_home_ interview.
Licensing. We do the Leetcode interview in a controlled testing center. When you apply for a position, I look up your license number, then I know you can leetcode without wasting any of my developer resources on determining that.
> Licensing. We do the Leetcode interview in a controlled testing center.
Congratulations, you are now a "Certified Leetcoder (tm)". :-(
Seriously: what a lot of people write down here is that a lot of programming jobs don't involve code puzzle skills, but are often rather about putting stuff/APIs together in the currently fashionable framework.
This makes becoming a Certified Leetcoder (tm) just another useless hoop to jump through.
(Just to be clear: for those few programming jobs that demand the employee to solve algorithmic puzzles regularly, doing them in a job interview makes sense. But these jobs are rare.)
> This makes becoming a Certified Leetcoder (tm) just another useless hoop to jump through.
And this differs from the status quo how? Employers obviously find value in this signal for better or worse. We’re just making it so it only needs to be done once, by trained proctors, instead of for every position you apply for.
Isn’t this basically triplebyte? I did their process and every company still wanted to do their leetcode interview after.
Former head of product there here: no, we didn't do this kind of identity verification. That would have prohibitively damaged people's willingness to actually do the interview. We did use various means to try to identify people signing up multiple times, and we caught plenty of people trying to duplicate themselves that way, but you didn't have to physically go to a center to do our interview.
I've considered something like that for my current company, which is doing basically the same thing, but:
(a) this has not, in practice, been a problem for us in identifying good candidates
and, far less importantly:
(b) you need very high scale to have interviewers everywhere that candidates are OR you're paying extra for a third-party controlled environment
(c) scheduling and cancellations become more difficult and costly respectively
When it comes to interviews I generally stick to asking fairly easy questions but leaving out key details, I care a lot more about candidates asking following ups and talking through the code they are writing over what code they actually produce. If a candidate can ask questions when they don't understand something and talk through their thought process then they are probably going to be a good person to work with. High level design questions are often pretty valuable I find as well, which I usually don't require code for I just ask them to talk through their ideas of how they would design an application.
In my uni days, I respected professors who designed exam in a way where students can utilize whatever they could to complete the assignment, including internet, their notes, calculators, etc.
I think the same applies to good tech interview. Company should adapt hiring process to friend with AI, not fight.
What would you suggest?
Ask questions that are tricky to cheat, evaluate candidates by their thought process in solving a problem and ability to clarify, discuss and justify their decisions.
nah ai killed stupid tech interviews. you can easily get an idea of someones competence by literally just talking to them instead of making them do silly homework exercises and testing their rote memorisation abilities.
This is the real answer. However to gauge competence you must first have it. The fact that most people don't is why we are in this position in the first place.
I don't think it did, if anyone cares. The way I've been advocating to my colleagues who are concerned about "cheating" is that there's probably a problem with the interview process. I prefer to focus on the think, rather than the solve.
Collaborate, as opposed to just do.
Things that really tell me if I can work with that person and if together, we can make good things.
It’s simple don’t have a tech interview that does not relate to the job.
Show code, ask questions about it that requires opinion.
This oft repeated talking point lacks perspective on what companies want and how interviews work. It also doesn’t address the AI problem.
Interviews are screening for multiple things, not just the ability to do one specific technical job. More often than not, technical coding ability is not even at the top of the priority list. Interviews are looking for well-rounded candidates who can do more than 1 job. Companies want to know if you can change jobs easily, they want to know if you’re average, better than the average programmer, or exceptional. They want to know if you’ll make a good manager after a few years, how good you are with people, how well you prioritize and communicate.
I had a professor in college that graded tests with the median skewed low, centered on a D. He complained that the usual practice of putting it on C or B made it so he could clearly see the difference between F-, F, and D- students, while the A students were all clumped together. He wanted to identify the hard workers and superstars in the class, see who was A vs A+ vs A++. It freaked everyone out when grades came out much lower than expected, but he renormalized at the end and people with test scores in the Cs and Ds got As and Bs in the class.
Be careful what you wish for. It’s competitive right now and interviews that limit screening to ability to do basic job-level coding and don’t screen for knowledge and soft skills and exceptionalism will make it harder for people who are good to demonstrate they’re better than people who are mediocre or use AI. Is that what you want?
Stop remote tech interviews
Unless the job you're interviewing for is remote-only, this makes perfect sense. If you expect your candidates to be able to work in your office, they should be interviewed there.
So, when AI can pass the tech interview seamlessly, I guess we can just hire it?
Maybe the future will be human shills pretending to be job candidates for shady AI “employment agencies” that are actually just (literally) skinning gpt6 apis that sockpuppet minimum wage developing nation”hosts”?
I think that a mythology about where the difficulty in working with computers lies has made the relationship between businesses and the people they hire to do this stuff miserable for quite some time
"Coding", as in writing stuff in programming languages with correct syntax that does the thing asked for in isolation, has always been a very dumb skill to test for. Even before we had stackoverflow syntactic issues were something you could get through by consulting a reference book or doing some trial and error with a repl or a compiler. That this is faster now with internet search and LLMs is good for everyone involved, but the fact that it's not what matters remains
The important part of every job that gets a computer to do a thing is a combination of two capabilities: Problem-solving, that is, understanding the intended outcome and having intuition about how to get there through whatever tools are available, and frustration tolerance: The ability to keep trying new* stuff until you get there
Businesses can then optimize for things like efficiency or working well with others once those constraints are met, but without those capabilities you simply can't do the job, so they're paramount. The problem with most dinky little coding interviews wasn't that you could "cheat", it's thst they basically never tested for those constraints by design, though some clever hiring people manage to tweak them to do so on an ad hoc basis sometimes
* important because a common frustration failure mode is repetitive behavior. Try something. Don't understand why it doesn't work. Get more frustrated. Try the same thing again. Repeat
thank fuck. they are terrible. being interviewed by CTOs just out of university, with no experience for a senior in everything role. they ask you to do some lame assignment, a pet problem not once looking at 20 years of GitHub repos and open source contributions.
Funny enough, the songs from the website Coding For Nothing about grinding LeetCode and endless take-home projects seem very relevant, and everything nowadays feels like a meme.
Tech interviewing has become a weird survival game, and now AI is flipping the rules again. If you need a laugh: https://codingfornothing.com
One option is to make the interviews harder and let candidates use ai to see how they can work with ai and actually build working product. They will be using ai in the job anyway so let them use it instead of asking stupid algorithm questions to sort an array
So there’s AI that’s really good at doing the skills we’re hiring for. We want you to not use AI so we can hire you for a job that we’re saying we’re going to replace with AI. Sounds like a great plan.
No. Using AI requires a depth of knowledge to spot the mistakes in the generated. code, and to know how to fit all the snippets of code in to something that works.
We need to know that the developer actually has skills and isn't just secretly copying the answer off of a hidden screen. We are interviewing now, and some cantidates are obviously cheating. Our interview process is not leet code based, and reasonably chill, but we will probably have to completely rethink the process.
Since we are hiring contractors, in theory we can let them go after a couple months if they suck, but we haven't tested out how this will work in practice.
The article actually takes a stance that if you incorporate AI you can learn how the person is at doing things the AI cannot.
Maybe we don't need employers. Maybe we need a bunch of 1-person companies. I don't think AI is yet the force multiplier that makes that feasible for the masses, but who knows what things will look like in a few years.
That is the end point of AI in the economy, until we are removed from the economy entirely.
Everyone on the planet is a one person startup, using AI and robotics to do all the actual work.
What's the point of humans (other than the ai and robotics ip owners) in this scenario?
Probably the same as it ever was. I couldn't tell you what that is, but it never had anything to do with economics anyway.
Having a job has always been a necessary evil. If the time is coming when that changes, all the better. If we can't figure out how to behave in the presence of plenty, we don't deserve utopia anyway.
You’re asking what is the point of being alive if you don’t have to work at a company?
Anyone with skill and agency needs capital, exposure, resources.
Companies currently provide this.
I think code design can often cover just as much as actual code anyway. Just describe to me how your solve it, the interfaces you'd use, and how you'd show me you solved it.
As an interviewee it's insane to me how many jobs I have not gotten because of some arbitrary coding problem. I can confidently say after having worked in this field for over a decade and at a FAANG that I am a very capable programmer. I am considered one of the best on every team I've been on. So they are definitely selecting the wrong people IMO.
> What are our options?
* Take a candidate's track record into account. Talk with them about it.
* Show that you're experienced yourself, by being able to tell something about what someone would be like to work with, by talking with them.
* Get a reputation for your company not tolerating dishonesty. If someone cheats in an interview and gets caught, they're banned there, all the interviewers will know, and the cheater might also start to get a reputation beyond that company. (Bonus: Company reputation for valuing honesty is attractive to people who don't want dishonest coworkers.)
* Treat people like a colleague, trying to assess whether it's a good match. You're not going to be perfectly aligned (e.g., the candidate or the company/role might be a bit out of the other's league right now), but to some degree you both want it to be a good match for both parties. Work as far as you can with that.
(Don't do this: Leetcode hazing, to establish the dynamic of them being there to dance for your approval, so hopefully they'll be negged, and will seek your approval, won't think critically about how competent and viable your self/team/company are, and will also be less likely to get uppity when you make a lowball offer. Which incidentally places the burden of rehearsing for Leetcode ritual performances upon the entire field, at huge cost.)
The death of shitty interviews has been greatly exaggerated.
AI might make e.g. your leetcode interview less predictive than it previously would have been. But was it predictive in the first place? I don't think most interviews are written by people thinking in those terms at all. If your method of interviewing never depended on data suggesting it actually, you know, worked in the first place, why would it matter if it starts working even worse?
Insofar as it makes the shittiness of those interviews more visible, the effect of AI is a good thing. An interview focused on recall of some specific algorithm was never predictive, it's just now predictive in a way that Generic Business Idiots can understand.
We frequently interview people who both (a) claim to have been in senior IC roles (not architect positions, roles where they are theoretically coding a lot) for many, many years and (b) cannot code their way out of a paper bag when presented with a problem that requires even a modicum of original reasoning. Some of that might be interview nerves, of course, but a lot of these people are not at all unconfident. They just...suck. And I wonder if what we're seeing is the downstream effects of Generic Business Idiots hiring primarily people who memorize stuff than people who build stuff.
> A lot of these people … just suck.
Another possibility is that their job subtly drifted.
I wrote a lot of code as a grad student but my first interviews afterward were disasters. Why? Because I’d spent the last few months writing my thesis and the few months before that writing a very specific kinds of code (signal processing, visualization) that were miles away from generic interview questions like “Make the longest palindrome.”
We don't ask "make the longest palindrome". We ask "convert this English into code that does what it says". If you want to make the discussion more concrete, we have a public practice problem [1] that we send out with our interview bookings so that people know what to expect. The real problems we ask are very similar to it.
Do you feel like there's anything there that any reasonably skilled programmer shouldn't be able to figure out on the fly?
[1] https://www.otherbranch.com/shared/practice-coding-problem
That's not a typical leetcode problem though. Most companies ask things like "solve this slightly modified knapsack problem" which takes 5 minutes if you know the solution and 50 minutes if you don't.
Yeah, it's very intentionally not a typical leetcode problem, precisely because leetcode is nearly worthless for screening people these days. I was replying to someone who seemed to be objecting to my opinion of some candidates' skills on the basis that maybe they just didn't know the specific thing we ask.
We did an experiment at interviewing.io a few months ago where we asked interviewees to try to cheat with AI, unbeknownst to their interviewers.
In parallel, we asked interviewers to use one of 3 question types: verbatim LeetCode questions, slightly modified LeetCode questions, and completely custom questions.
The full writeup is here: https://interviewing.io/blog/how-hard-is-it-to-cheat-with-ch...
TL;DR:
- Interviewers couldn't tell when candidates were cheating at all
- Both verbatim and slightly modified LeetCode questions were really easy to game with AI
- Custom questions were not gamable, on the other hand[1]
So, at least for now, my advice is that companies put more effort into coming up with questions that are unique to them. It's better for candidates because they get better signal about the work, it reduces the value asymmetry (companies have to put effort into their process instead of just grabbing questions from LeetCode etc), and it's better for employers (higher signal from the interview).
[1] This may change with the advent of better models
A couple years ago, we were using a take home coding assignment as a hiring signal. It was a small API based off something I'd built for an internal tool. It was self-contained and relatively easy to explain. The .md file was about two pages.
I recently fed it into ChatGPT and asked it to do the assignment. It did it perfectly -- I read the code in detail and couldn't find any issues.
So custom questions are off the table now, too. We'll be using a code review instead for the next round.
The inconvenient truth is that everything circles back to in-person interviews.
The article addresses this:
>A lot of companies are doing RTO, but even companies that are 100% in-office still interview candidates from other cities. Spending money to fly every candidate out without an aggressive pre-screen is too wasteful.
No, accidently hiring someone who AI'd their way through the interview costs orders of magnitude more to undo. It's absolutely worth paying for a round trip flight and a couple days of accommodations.
1point3acres is massacring tech interviews right now. Having to pay $80/month to some China based website where NDA-protected interview questions are posted regularly, then being asked the same questions in the interview, seems insane.
It also feels like interviewers know this and assume you studied the questions, they seem incapable of giving hints, etc if you don't have the questions memorized.
AI is the least of it.
Very funny :) I too failed an interview at google, also related to binary search on a white board. I never write with pens. I'm on keyboards the whole time, my handwriting is terrible.
I've built a search engine for two countries and then I was failed by a guy that wears cowboy hats to work at google in Ireland. Not a lot of cows there I'm guessing. (No offence to any real cowboys that work at google of course).
I did like the free flight to Ireland though and the nice lunch. Though I was disappointed I lost "Do no evil" company booklet.
TBH, not the #1 source of animal meat, though plenty of cows in Ireland.
Good for google. Plenty of passable interview candidates then.
Dang! I knew it was a mistake leaving my hat at home. Little things like that people tend to forget.
The best interview process I've ever had was going to work with former coworkers, aka no real process. A couple of quick calls with new people who deferred strongly to the person who knew me, my work, and my values. Nothing else has the signal value.
Of course the problem is this can't scale or be outsourced to HR, but is this a bug or a feature?
The best interview processes are chill, laid back, open ended.
That's the only way you're going to get relevant information.
I've been verified to the moon and back by Apple and others for roles that could never have worked.
The problem is that when it comes to the hiring process, everyone is suddenly an expert; no matter how dysfunctional, inhumane and destructive their ideas are.
Anyone who suggests a paired programming solution is right, and answering the wrong question. Unless/until we return to a covid-like market the process will never be optimized for the candidate, and this is just too expensive an approach for employers. In this market I think the answer is hire less.
one hiring manager told me they dont do code challenges. they said, "why would someone take a job they couldnt do?"
isnt it that simple?
If using AI is cheating then one solution as the author mentions is have the interview take place at an office but I'm surprised another approach isn't more readily available: having the candidate the the test remotely at a trusted 3rd party location.
I just ask to share a text editor and write down my questions. Its critical anyway because often then not its not always clear for tech questions what exactly i asked (linux command for example).
This blocks their screen too.
and yes we do know very soon if you look somewere else, take time or rephrase the question to get more time.
If you able to fake it, at that point you should just get th ejob anyway :P
Why don't we simply ask the AI how to conduct a tech interview nowadays?
Interestingly I find AI is actually better at that kind of CS whiteboard question (implementing a binary search tree) than that "connecting middlewares to API" type task. Or at least, it's more straightforward to apply the AI, when you want a set of functions with clearly defined interfaces written - rather than making a bunch of changes across existing files.
I’ve wondered about the kind of person who starts white boarding with the pros and cons of several AI offerings. As if, confronted with a problem domain, they are choosing or hiring AI before architecture. As an interviewer, how should I adapt my questions? Something like, “How would you prompt it to add fuzzing?” “How would you prompt it to describe how each change might affect our stack?”
If you are using Internet, Google, Stackoverflow at work, why insist that interviewee need to solve problems on their own?
To test their inherent thinking skills and base knowledge.
There's a huge difference between occasionally looking up something, and practically leaning on it. Ironically, the mass degradation of search engine result quality within the past ~decade has made it much harder for people to do the latter, and when they do, it shows much more clearly.
Don't forget to wear your cowboy hat when interviewing at google. Very important.
I've been considering using a second webcam stream focused on my screen just to assure hiring managers that I don't have ChatGPT on my screen, or anywhere else. Kind of like chess players do it sometimes on online tournaments. I've been hearing people complain about cheating a lot.
Ask for an in-person interview instead of voluntarily encouraging more mass surveillance.
But what if I bring my AI butt plug?
Company wide policy, apparently.
I've been interviewing a bunch of developers the past year or so, and this:
> Architectural interviews are likely safe for a few years yet. From talking to people who have run these, it’s evident that someone is using AI. They often stop with long pauses, do not quite explain things succinctly, and do not understand the questions well enough to prompt the correct answer. As AI gets better (and faster), this will likely follow the same fate as the rest but I would give it some years yet.
Completely matches my experience. I don't do leet code BS, just "let's have a talk". I ask you questions about things you tell me you know about, and things I expect of someone at the level you're selling yourself at. The longest it's taken me to detect one of these scumbags was 15 minutes, and an extra 5 minutes to make sure.
Some of them make mistakes that are beyond stupid, like identity theft of someone who was born, raised and graduated in a country whose main language they cannot speak.
The smartest ones either do not know when to stop answering your questions with perfect answers (they just do not know what they're supposed to not know), or fumble their delivery and end up looking like unauthentic puppets. You just keep grinding them until you catch em.
I'm sure it's not infallible, but that's inherent to hiring. The only problem with this is cost, you're going to need a senior+ dev running the interview, and IME most are not happy to do so. But this might just be what the price of admission for running a hiring pipeline for software devs is nowadays. Heck, now feels like a good time to start a recruitment process outsourcing biz focused on the software industry.
I think this approach is not very favoured by hacker news but it's also what I prefer. It's so much easier to quickly gauge a minimum level of the basic programming knowledge and other sw knowledge by just asking some simple directed questions.
I once got a guy that claimed to have implemented multiple default HTTP JSON REST APIs and somehow had never:
- tested his API with JSON payloads, serialise serialise - never queried his APIs manually or semi automatically (no knowledge of curl, postman or anything similar)
I never understood why Big Tech never setup contracts with all the SAT and ACT test centers across the country. Even before Zoom with Codepads it would have made sense for the recruiters to send potential candidates to a test center to do a pre assessment rather than waste time with engineers sitting on prescreen calls all day.
What exactly are you suggesting here? A standardized test that applies to all your job applications? Or, a candidate having to drive to a test center for every company they apply to? Or something else?
the idea is sound. create a basic standardized test targeted at tech/engineering jobs. not actually SAT -- operated by a vendor like The College Board. There are plenty of standardized test operators
When I'm interviewing, I'm putting about 30% of the weight towards "would I enjoy working with this person on a daily basis?", but in the context of technical discussions. Standardized testing won't be able to replicate it.
you're not allowed to discriminate
Discriminate against... a personality that will negatively impact the team dynamics? It's not that easy, to be honest, as every team has its own requirements.
A stereotypical Asian interviewing a stereotypical German might find the German rude in some interactions. While another German interviewer would find it being frank.
Interviews based on personal feelings have hidden biases not even the interviewer is aware of.
Here's another question - stereotypical Japanese interviewer, interviewing, back-to-back, a stereotypical Indian and a stereotypical German for the role. Both are capable and equally technically proficient. How do you choose, other than looking at the team you're hiring for, and thinking how the person would fit in?
would you be able to document this negative fit in an impartial way when rejecting a protected class?
You just send out a generic decline, and document it as there's a better candidate fit for the role.
I'm not sure if you guys have been in charge of hiring, but there's no real alternative. In my most recent experience, we had one open position, and after interviewing 10 candidates, 3 of them were basically identical in terms of technical qualifications. How do you choose one over the other, other than the "vibes"? Anyone suggesting otherwise is either living in a weird alternate reality, or doesn't want to accept that working is a cooperative job and interpersonal relationships are very important.
There always will be exceptions for different type of roles and specializations, but that's not what I'm talking about.
a large company doing this (no documentation of a skills gap) who gets subpoenaed would lose 10/10 times
I'm curious, what are the recent cases that were brought up where the company lost?
Meta had one around 2020 with EEOC. The Harvard Case I mentioned above (supreme court) , Activision, Dell, a few others on the top of my mind.
> would you be able to document this negative fit in an impartial way when rejecting a protected class?
Likely. But haters gonna hate, and lawyers gonna sue.
Is that discrimination? Somebody can en an annoying prick regardless of their background.
If the candidate is a protected class and they are rejected for "cultural fit" it will be an easy case for EEOC to raise a discrimination case.
This is effectively how Harvard was rejecting Asian applicants. They created a "personal fit" / cultural fit quality that Asians scored low on . Supreme Court found this to be discrimination.
It doesn't matter if you are truly discriminating, it matters how well you have tangible evidence of the employee not meeting the qualifications for the role.
Would certainly make it easy for employers to differentiate themselves by saying “well, at least we don’t do THAT”.
Interviewing fads are set by these large companies that have the problem of systematically evaluating many thousands of candidates. A standardized test is all they want. Then they could do the rest like college admissions, and at a fraction of the cost.
This largely misses what an interview is all about, save for entry level positions.
given the in-person interview is at the end of the funnel by a factor of 500-1000, standardized testing might even open up opportunity for under represented candidates.
Think of how poor the screening process is at the recruiter & CTS (left side) of the funnel, and how many false negatives there are .
If you could offer standardized test at that level, you may be able to keep viable candidates in the funnel longer.
I don’t think that the suggestion is that the standarised test is the only filter. It is merely one out of many.
That’s what Triplebyte planned on. The truth is I don’t trust anyone else to run evals.
you're right there are quality issues, some probably deliberate.
but the screening cost for companies is eye watering so something should be done.
> The truth is I don’t trust anyone else to run evals.
It's a common sentiment.
But compare https://www.cambridge.org/core/journals/judgment-and-decisio... . ("People predicting the future performance of college students state that interviewing the students aids prediction, although in fact the interviews make predictions less accurate.")
Well, the people who read that are welcome to try that and outcompete me in the market. Triplebyte still exists.
Standardized plus the ability for companies to do their own test after they pass the standard one. So go get prescreened at test center then use that test to apply for jobs. Company either flys you in for in-person or sends you back to test center to do live remote interview in controlled environment.
It's been tried and failed: Sun Microsystems pushed certifications in the 90s. Pass the test on some technology, get the certification. Then they studied performance. The result? More certifications implied a worse employee. The reason was the top performing employees had no time to study for the exams, but the managers of the bottom performing employees were happy to send them off to training and testing. And then the certification fad came mostly to an end.
That was something quite different that got tried. This would be more based on aptitude rather than knowledge.
Of course, they'd miss out on some good talent. But in the article where it shows the quote of someone getting rejected for not inverting a binary tree on a whiteboard, that doesn't seem like a terrible thing to test for.
I always felt like the Sun and Cisco certs were more about creating people that would push their products on other companies.
Big Tech / Unicorn / Wannabe Unicorn prescreens are all basically standardized now anyway.
That was essentially triplebyte.
That was part of it. Another part was it being socially awkward in a way that I think outsourcing testing needn't be.
No, that was Triplebyte's marketing. Their actual product was completely different.
I have done interviews with Karat which just outsources technical interviews to engineers elsewhere.
All of these technical interviews still suck though! I basically never code with someone watching me and find it very difficult to do in interviews. I also find it hard to find the time to actually practice this skill
Im revisiting technical interview prep because it has been awhile and it seems like a good time for a refresher, and it is striking how similar it all is to SAT and GMAT prep these days. A pretty cookie-cutter performance that is mostly about demonstrating that you have the time and means to properly prepare. Might as well just go the extra step at that point and have it be exactly like those standardized tests… take them once at a test center, get a score that is valid for a few years that you can just send in with your application…
That maybe end up happening. Send them to a test center to do their remote hacker rank interview. Doesn’t even need to be standardized. I don’t like it but it’s one of the lower friction options.
In the US and Canada, to get into university, there is, or at least used to be, a standard entrance exam or something close to it (the SAT in the US, OAC scores if you were from Ontario, Canada, etc..).
Additionally, Undergraduate programs in the US and Canada, at least used to, despite their varied reputation, have a pretty standard program.
Maybe things have deteriorated so far at the high school and university level that new standardized exams are needed. But we also have a plethora of verifiable certifications whose exams are held in independent test facilities.
Another commenter on this called it “Licensing” but it’s more like credentialing to me.
I miss one option from the list of non-solutions the author presents there - ditch the idiotic whiteboard/"coding exercise" interview style. Voila, the AI (non)problem solved!
This sort of comp-sci style exam with quizzes and what not maybe somewhat helps when hiring junior with zero experience fresh out of school.
But why are people with 20+ years of easily verifiable experience (picking up a phone and asking for references is still a thing!) being asked to invert trees and implement stuff like quicksort or some contrived BS assignment the interviewer uses to boost their own ego but with zero relevance to the day to day job they will be doing?
Why are we still wasting time with this? Why is always the default the assumption there that the applicants are all crooked hochstaplers that are lying on their resumes?
99% of jobs come with probationary period anyway where the person can be fired on the spot without justification or any strings attached. That should be more than enough time to see whether the person knows their stuff or not after having passed one or two rounds of oral interviews.
It is good enough for literally every other job - except for software engineering. What makes us the special snowflakes that people are being asked to put up with this crap?
No, it has not. We still have the same situation.
Never really liked leetbro interviews. Always reeked of “SO YOU THINK YOU CAN CODE BRO? SHOW ME WHAT YOU GOT!” The majority of my work over 10+ years of experience always relied on general problem solving and soft skills like collaborating with others. Not rote memorization of in order traversal.
> Tech interviews are one of the worst parts of the process and are pretty much universally hated by the people taking them.
True.
> One of the things we can do, however, is change the nature of the interviews themselves. Coding interviews today are quite basic, anywhere from FizzBuzz, to building a calculator. With AI assistants, we could expand this 10x and have people build complete applications. I think a single, longer interview (2 hours) that mixes architecture and coding will probably be the way to go.
Oh.... yeah, that sounds just... great.
"One of the things we can do, however, is change the nature of the interviews themselves. Coding interviews today are quite basic, anywhere from FizzBuzz, to building a calculator. With AI assistants, we could expand this 10x and have people build complete applications. I think a single, longer interview (2 hours) that mixes architecture and coding will probably be the way to go."
If that's where the things are going, I'm retraining to become a line cook at McDonalds.
> I think the image below pretty much sums it up
The image below does sum it up but not in the way the author thinks.
Google wants to hire people who complete their hiring process. They're OK with missing out on some people who would be excellent but who can't/won't make it through their hiring process.
The mistake may lie in copying Google's hiring process.
None of this makes any sense. Why should I complete a tech test interview if I have 15 years of experience at X top firm? I would have done it already anyway.
I had a ‘principal engineer’ at last place who grinded leetcode for 100 days and still failed a leetcode interview. It’s utter nonsense.
A conversation with technical questions and topics should suffice. Hire fast and fire people.
LLMs killed busy work. Now people have to actually talk to each other and they're finding out that we've been imitating functionality instead being functional.
What a BS article. As they say, just do the interview in person. Problem solved. Not sure about the US but 99% of jobs here in Spain are hybrid or onsite ("presencial"), not fully remote.
They're acting like all jobs are remote and it's impossible to do an interview in person.
Also, does it really matter? If a person is good at using AI and manages to be good at creating code with that, is it really so much worse than a person that does it from the top of their head? I think we have to drop the idea that AI is going to go away. I know it's all overhyped right now but there is definitely something to it. I think it will be another tool in our toolboxes. Just like stackoverflow has been for ages (and that didn't kill interviews either).
Our US mega corp is having us hire all remote contractors, in person is completely out of the question.
Costs money (travel) and time (company's) plus a lot of this is outsourced to agencies to do the screening.
Yes, it is disgusting. Sadly also very common.
Ah weird, here in Europe if you apply for a job in another city or country they won't fly you out there. You can come on your own dime and usually you wouldn't even tell them you don't live there yet (after all if you don't even live there, why would they bother with you, it's only extra hassle for them when you start). Probably C-suite roles are an exception to this. But they're an exception to pretty much everything. Roles with particular foreign languages (e.g. support) too but this is also an edge case.
Moving personnel between countries when they are already working for the company does happen. They did it for me. But at that point they already know what they have.
[dead]
It hasn't killed the interview, it's killed the career field. Most people just haven't realized this yet.
"Video killed the radio star"
Show the remote candidate an AI's deficient answer to a well-asked question, and ask the candidate if they understand what exactly is wrong with the AI's assessment, or what the follow-up/rewritten prompt to the AI should be. Compile a library of such deficient chats with the AI.
AIs are probably good at answering such questions.
here's the question, here's the code that ChatGPT produced. what's wrong with it?
It's a tricky subject, because what if people who use AI are just better together? And what if in a year from now, AI by itself is better? What's the point of hiring anyone? Perhaps this is the issue behind the problems being described, which might be mere symptoms. There are tons of very smart teams working on software that will basically replace the people you're hiring.
Or you can just given them a way to bypass all of that, and ask them about any significant project that the candidate did build (which is relevant to the job description, open or closed source that is released) or even open source contributions towards widely used and significant projects. (Not hello world, or demo projects, or README changes.)
Both scenarios are easily verifiable (can check that you released the project or if you made that commit or not) and in the case of open-source, the interviewer can lookup at how you code-review with others, and how you respond and reason about the code review comments of others all in public to see if you actually understand the patches you or another person submitted.
A conversation can be started around it and eliminates 95% of frauds. If the candidate cannot answer this, then no choice but give a leetcode / hackerrank hard challenge and interview them again to explain their solution and why.
A net positive to everyone and all it takes to qualify is to build something that you can point to or contribute to a significant open source project. Unlike Hackerrank which has now become a negative sum race to the bottom quest with rampant cheating thanks to LLMs.
After that, a simple whiteboard challenge and that is it.
This would be a nice interview for candidates who have open source contributions, but many who have day jobs do not. Or their open source code is 5 years old and not representative of their current skill set.
There is no shame in taking time off after leaving a job to develop or contribute to an open source project or two. The world would be a better place for it.
I don’t think shame is the obstacle; it’s that for many, a break like this is financially straining or outright impossible.
That is a lot like saying that a college degree is financially straining or outright impossible. In many respects, developing open source is a lot less straining, as there are no large fees, with the main expenses being living expenses. This is why it's important to live far below one's means when one does have a job.
That is a fair point.
Well the reason I don’t has nothing to do with shame and everything to do with time. I’m allocating my extra time to work, which (on a good day) makes my company money.
For a candidate who does have OS contributions, that’s great but most will not. And the more senior they are the less likely I would imagine.
If you already have a job, you don't strictly need a different one. If you really need one, it should be okay to quit the old one first, perhaps work on open source as needed, then get a new one. As it is often said, looking for a job is a full-time job.
Come on - it was already dead for a long time.
I cannot emphasize this enough. Coding is the EASY part of writing software. You can teach someone to code in a couple of months. Interviews that focus on someone's ability to code are just dumb.
What you need to do is see how well they can design before writing software. What is their process for designing the software they make? Can they architect it correctly? How do they capture user's mental models? How do they deal with the many "tops" that software has?
No it didn't, you just need to stop asking questions an LLM can easily solve, most of those were probably terrible questions to begin with.
I can create a simple project with 20 files, where you would need to check almost all of them to understand the problem you need to solve, good luck feeding that into an LLM.
Maybe you have some sneaky script or IDE integration that does this for you, fine, I'll just generate a class with 200 useless fields to exhaust your LLM's context length.
Or I can just share my screen and ask you to help me debug an issue.
I know nobody likes doing tech interviews but how has AI killed it ? Anyways you do want to know basics of computer science, it is a helpful thing to know if you ever want to progress beyond CRUD shitshovelling.
Also wtf is inverting a binary tree ? Like doing a "bottom-view". That shit is easy.
[dead]