How to Obtain Detailed Traces for Chatbot Created with Ragflow?

[erwan-simon-CATIE]

Hello everyone,

I’m currently exploring Ragflow and I’m interested in capturing detailed traces of the chatbot's process at each stage. The goal is to thoroughly analyze how the system behaves.

The kind of stuff that interest me are:

• The prompt passed to the LLM at the final stage (and idealy, at each steep of the pipeline)
• How the context is modified by the reranker (ei. have chunk list before and after for comparaison)
• ... Any other intermediary steps that contribute to the final output ! :-)

To achieve this, I was considering the possibility of connecting Ragflow to a Langfuse instance, which seems like a promising solution for trace logging and monitoring. However, I'm unsure about the best way to proceed.

Here are a few specific questions I have:

• Is there any built-in functionality within Ragflow that allows capturing such detailed process traces?
• Has anyone attempted to integrate Ragflow with Langfuse (or a similar tracing tool)? If so, how can this be done effectively?
• Are there any recommended best practices or tools to track the flow, especially for analyzing how prompts evolve and how decisions are made at each stage?
• Any insights or suggestions on this would be highly appreciated. Looking forward to your feedback!

Thank you in advance.

Best regards,

[erwan-simon-CATIE]

For anyone looking for a way to access the query trace (for testing purposes)... just click on the light bulb icon :-)

[KevinHuSh]

The 'retrival test' in knowledgebase can also help you to understand the data before sending to llm.