The accuracy of LLM responses is crucial when implementing AI solutions for your business. GraphRAG solutions have proven to be effective at achieving critical accuracy rates.
Self-service functions in use cases such as customer support can greatly reduce costs and increase revenue. GenAI applications powered by Large Language Models (LLMs) are leading the way in leveraging knowledge bases to provide accurate information to customers.
Many teams are already leveraging Retrieval-Augmented Generation (RAG) to improve the accuracy and timeliness of the responses they receive from their LLMs. RAG enhances an LLM’s general-purpose data with more up-to-date and domain-specific supplementary data. This might include a product catalog, internal documentation on a product, support ticket information, etc.
But can a graph-based RAG solution provide even greater accuracy? In this article, we’ll look at how graph-based RAG solutions have proven effective at achieving critical accuracy rates and share some lessons we’ve learned that are applicable across a number of domains.
A lot of the work that we do at Agolo in terms of mining knowledge bases is centered on providing self-service support for customer service via modalities such as chat. It’s a domain where accuracy is crucial, as providing the wrong answer is often incredibly expensive.
For example, data from Forrester Research shows that it costs a mere $0.10 per self-service support contact versus $12 for a call center support contact. In other words, providing the wrong answer is 120 times more costly.
However, there’s also a long-term cost to offering the wrong answer. Other studies have shown that 84% of businesses that improved online customer service saw an increase in revenue. Improved self-service also increases customer retention by around 5%. That leads to an increase in profits between 25% to 95%.
According to a survey by LivePerson:
The challenge is that, while businesses have high hopes for GenAI, customer trust in GenAI solutions is low. Salesforce found that 54% of users don’t trust the data used to train AI systems.
Why is customer trust so low? A team of researchers at Stanford dug into this. They found that roughly 28% of answers from RAG-powered GenAI tools contained hallucinations. Roughly the same amount of answers came back incomplete.
The underlying root cause lies in the nature of LLMs themselves. LLMs are designed and very effective at answering general questions based on general knowledge. They’re designed to answer those in a common language that is understandable by most people, not in specialist-speak.
In other words, when providing product support - either via self-service, to agents, or to field technicians - we’re asking LLMs to answer very specific questions that require specific knowledge and specific terminology. That's why we’re seeing higher hallucination and incomplete answer rates. That, in turn, results in a loss of trust and lower adoption rates by users - whether customers, agents, or field technicians.
At Agolo, we’ve found that customers come to trust a GenAI solution in a field like support when they are confident that the AI application:
RAG data is often stored in a vector database and queried before a prompt is sent to an LLM. For the rest of this article, I’ll refer to this as “baseline RAG.”
We’ve found that baseline RAG works well under a few limited conditions:
Sure, you can add more articles to your knowledge base to increase the context window. But you’re not really describing how these relate to one another or how they relate to your products.
For example, your system doesn’t understand which articles relate to a product family vs. a specific model of your product.
Agolo’s GraphRAG goes a step beyond baseline RAG by supplying an LLM a graph of information related to the question at hand. This enables taking into account information such as your product taxonomies and the relationships between products, issues, and content.
GraphRAG solves this problem by improving on RAG in five aspects:
1) Normalization of similar names. We start by extracting entities and linking products, parts, issues, and other entities together. We then resolve things such as name variations, partial names, misspellings, noise, and the link.
2) Knowledge graph generation. With all entities normalized, we automatically generate a graph of relationships between identities, identifying relationships between (for example) products, parts, and issues. The goal is to tie your graph back to your taxonomy of products and issues. This creates a Dewey Decimal-like structure that makes it easier to discover material, as similar assets are co-located.
3) Traceability. The knowledge graph we generate includes data lineage with links back to source documentation. That means we can provide the full context for any answer - not just text, but also images, diagrams, videos, and other multimodal assets.
This generation of entities and traceability works on structured, unstructured, and semi-structured data. For example, when you have a table in a document with rows and columns, the headers of each column have meaning. A GraphRAG implementation can use that meaning to provide additional context and traceability.
4) Improved RAG. All of these factors lead to an improved, more accurate result when a user submits a question. We can pull out what entity they’re talking about and use a fine-tuned LLM to extract the right information and provide the most relevant results.
5) Integration into your existing RAG pipeline.All of this is easily integrated into your existing RAG pipeline. We provide a UI for managing GraphRAG, as well as an API you can use to automate ingestion (which is what most of our customers use).
Having run GraphRAG solutions now for a number of customers, we’ve developed a few principles that are critical to a successful project:
GraphRAG works. We’ve seen GraphRAG yield impressive results for our customers. This is due primarily to the power of taxonomy and organization, which adds a level of structure and context you don’t get with vanilla RAG.
Take a pragmatic approach. A lot of AI solutions are trying to use the “easy button.” Do a primitive search, pass it to ChatGPT, and call it a day. The pragmatic approach, in our way of thinking, is to make the right amount of upfront investment in your data - but without overthinking it.
This sounds simple. However, the devil’s in the details.
Internal teams or infrastructure vendors may think they can do this. But it requires a lot of time in the trenches - tailoring knowledge extraction, mapping it correctly to taxonomies, getting the right data model, and fine-tuning the question/response engine to get the best combination of speed and accuracy.
Learn from mistakes. One Agolo team member spent a long time at Amazon. One thing Amazon’s customer support was focused on was asking, not “Did we give you correct information?” but “Did we solve your problem?”
That’s a useful feedback mechanism that leads you to dig deeper into why a given interaction didn’t solve a problem. It could be an AI hallucination. Or maybe it’s missing information in your product taxonomy.
In Agolo, we manage this feedback loop through various mechanisms. We use a combination of human in the loop feedback as well as solutions that can automate such feedback loops.
Once you’ve eliminated hallucinations, user adoption will skyrocket. If your solution gets poor results, customers and customer service agents will get frustrated and give up on it. However, once you have a highly successful question/response rate, you’ll see user adoption climb very quickly.
Any GenAI solution can benefit from a mix of structured, unstructured, and semi-structured data. GraphRAG goes beyond baseline RAG in adding additional enrichment, context, and connection to this data. The result is greater accuracy and adoption than you could ever achieve using an LLM with or without pure RAG.
Want to learn more about how to bring GraphRAG to bear on your company’s hardest problems? Contact us today for a demo.
