A developer who can get a RAG demo running in an afternoon is easy to find. One who can explain why the system gave a wrong answer, how often it does so, and what to change first is much rarer. If you plan to hire RAG developer talent, the task is to tell the two apart before the contract is signed.

This article is written for the person doing the hiring. It covers the skills that matter, how to check a portfolio, practical tasks that reveal real ability, and the engagement options to choose between.

Skills That Matter

Tool names change quickly, so look for evidence of reasoning instead of a list of products. Familiarity with your own stack is a convenience, not a proxy for judgement. These qualities tend to separate strong candidates from weak ones.

  1. Retrieval judgement. They can explain why a passage was or was not found, and what they would adjust.
  1. Data sense. They have a method for messy material such as scanned files, tables, and duplicate versions.
  1. Evaluation habit. They measure quality with a test set and do not rely on how answers feel.
  1. Security awareness. They think about permissions and about what should never enter the index.
  1. Honesty about limits. They will tell you when RAG is the wrong tool or when your content is not ready.
  1. Plain communication. They can explain trade-offs to someone who is not technical.

Portfolio Checks

A polished demo shows the best case. To see more, ask for specifics.

What was the source material, and how messy was it? How did they measure accuracy, and what were the results? What failed, and what did they change as a result? Who used the system, and for how long? What was their own role, as opposed to the wider team's?

Ask for a live walkthrough rather than a recording, and ask questions the candidate has not prepared for. Where confidentiality limits what they can show, ask them to describe the approach and the problems encountered, without revealing client data. Ask, too, what happened after launch: how content changes were handled and who maintained the system. A candidate who can discuss failures in detail has usually done the work. One who only describes successes may not have.

Check claims independently where you can. A short conversation with a former client often reveals more than any written summary.

Seniority and Scope

Decide what level you need before you start looking. A system that will carry customer-facing answers needs someone who has seen RAG fail in practice and can design around it.

A well-defined extension to an existing system may suit a less experienced engineer, provided a senior person reviews the work. If the budget limits you to one hire, prefer judgement over speed, because poor early decisions about data and evaluation are expensive to reverse.

Practical Tasks

A short, time-boxed exercise can reveal more than a long interview. Consider paying for it, which signals respect for the candidate's time.

Diagnose a Failure

Give them a small RAG setup that returns poor answers, along with sample questions. Ask them to find the cause. Their method matters more than the outcome: do they check retrieval first, or guess at the model?

Design an Evaluation

Ask how they would test a system built on your documents. Look for a question set from real queries, separate scoring of retrieval and answers, questions that have no answer in the content, and a plan to re-run tests after every change.

Review a Document Set

Hand over a handful of your typical documents, including an awkward one. Ask how they would prepare them: how to divide them, what details to attach, what to exclude.

Threat-Model the Bot

Describe a scenario where users have different access levels. Ask what could go wrong and how they would prevent it. Good answers cover permissions, logging, and the risk of untrusted content in the index.

Interview Questions

A few direct questions go a long way:

  1. Describe a RAG system that gave poor answers. What was the cause?
  1. How do you decide how to split documents?
  1. How would you know that a change made things better?
  1. What would you refuse to put into the index?
  1. When would you advise against using RAG?

Listen for specific examples and for willingness to say "it depends, and here is on what". Be wary of answers that offer one technique for every situation.

Engagement Options

You have several routes, and the right one depends on whether you have someone internally to guide the work.

  • Freelance contractor. Flexible and quick to start. Continuity and availability depend on one person, and you carry the management load.
  • Dedicated developer through a firm. A named engineer with a company behind them, which can help with continuity and replacement. You still direct the daily work.
  • Project team. A group with defined deliverables and an agreed outcome. It reduces your management effort but needs a clear brief.
  • Hybrid. Your own lead working with external engineers. It works well when you have the product knowledge and need engineering capacity.

If you need an outcome and not a person, such as a working system with testing and handover, RAG development services from a firm may suit better than hiring an individual. Build in a trial milestone, such as a defined first deliverable reviewed after a few weeks, so that either side can change course without difficulty. Whichever route you take, agree access arrangements, documentation expectations, and what happens when the engagement ends.

Setting Up the First Month

A good start reduces the risk of a poor hire. Give the new developer access to a sample of your real content and a first small deliverable, such as an evaluation set for one use case. Review progress at fixed points in the first weeks, and ask them to explain their choices as well as show results.

Keep ownership of the evaluation set on your side, so that quality is always judged against your standards. Require documentation from the start, covering how content is prepared, how the system is configured, and how it is tested.

Avoid Red Flags

Be cautious if a candidate promises that answers will always be accurate, cannot describe how they measure quality, avoids questions about failures, or has no view on security.

Reluctance to take a practical task is also telling. So is a candidate who describes every project as a success and cannot name one thing they would do differently. Pay attention to how they handle questions they cannot answer: a candid "I do not know, but this is how I would find out" is a good sign.

If you are looking for RAG expertise and want to discuss your project with a reliable service provider, contact Amenity Technologies to talk through your requirements.

Frequently Asked Questions

What should I look for when I hire a RAG developer?

Evidence of reasoning: they can explain why retrieval succeeded or failed, they measure quality with a test set, they think about security, and they are honest about limits. Tool names matter less.

How do I assess a RAG developer's portfolio?

Ask about the source material, how accuracy was measured, what failed and what happened after launch. Request a live walkthrough and, where possible, speak to a former client.

Should I hire a freelancer, a dedicated developer or a project team?

It depends on whether you have someone internally to direct the work. A freelancer or dedicated developer suits that case, while a project team suits a defined outcome with less of your management.