As I understand it, RAG is focused more on capturing and using external resources on an ad hoc basis, where fine tuning looks to embed a specific 'behaviour' change in the model for a particular need?

You would need both. RAG will have to be part of the process if you want to generate or work on an existing project. Since it stands for Retrieval Augmented Generation. A fancy way of saying an LLM can access documents.

If you are strapped for time maybe try https://www.cursor.com/ or similar offerings that do some of this for you.

Chatting with GPT4 and pasting into your editor code or pasting into a terminal some commands could also help alot.

GPT-4 is the best coding model by far and you can’t fine tune it yet. Start with GPT-4 and RAG.

Fine tune is good for changing behavior. For example you could create a dataset that makes the model directly output fully formed projects. Using RAG alone would require a bunch of prompt engineering to get the same.