Scripting (like Harpa AI) to enabled advanced RAG

I would love to be able to create scripts to rewrite or augment prompts or increase interactivity with the user.

For example:

  1. keyword extraction
  2. rephrasing
  3. synonym substitution
  4. Hypothetical document embeddings (HyDE)
  5. prompt (re)compression

For a nice list of techniques, [see here] .(https://decodingml.substack.com/p/your-rag-is-wrong-heres-how-to-fix?publication_id=1526003&post_id=153581453&isFreemail=true&r=lob3f)

The book is even better if you're looking for techniques to improve Socrates' retrieval performance. I don't know how much Socrates is doing already, compared to other RAG SaaS providers.

But some of the techniques can be implemented using some kind of scripting system and that would take the development burden off of your engineers.