Dify

Introduction

The term "hallucination ¹" has become increasingly common recently. In the field of generative AI, hallucination refers to the phenomenon where large language models generate information that is not factual but appears plausible. I'm sure many of you have experienced generative AI confidently providing an answer that was actually incorrect. I certainly have, quite often.

One of the countermeasures for this hallucination problem is a technique called RAG, and Dify is one of the services that implements RAG. I will explain each of these step-by-step.

Causes of Hallucination

First, why do hallucinations occur?
The following are some of the causes of hallucination:

• Learning Data Issues
  AI learns from vast amounts of data, often from the internet, but not all information on the internet is accurate. It may learn misinformation or fiction as correct knowledge.

• Generation Process Issues
  Large language models generate text by statistically predicting the "next word." Consequently, they may generate sentences that are "contextually natural and plausible," even if they are factually incorrect.

• Fixed Knowledge
  They embed pre-learned knowledge internally. Information is not updated even if it becomes old or incorrect.

• Input Prompt Issues
  Even if instructions are ambiguous or content is insufficient, generative AI will make assumptions to complete the information. If this completion is inappropriate, it manifests as a hallucination.

Means to Address the "Fixed Knowledge" Problem

What are the ways to address the "fixed knowledge" problem?

• Retraining
  Retrain the model by adding the latest data. This is costly for large models and cannot be done frequently.

• Additional Learning (Fine-tuning)
  Add learning for specific domains or the latest information only. This is less costly than retraining, but still involves significant expense.

• Referencing External Information
  Generate answers by referencing external information in real-time. Recent generative AIs have started performing web searches.

Although it might seem that the "fixed knowledge" problem has been resolved now that generative AI performs web searches, there are still challenges such as the following:

• Web searches carry the risk of sourcing pages that contain misinformation or biased information.
• There is information that does not exist on the web or cannot be published on the web (e.g., internal company documents) that organizations wish to utilize internally.
• The desire to generate more accurate answers by leveraging documents and knowledge bases.

To solve these challenges, the RAG technique emerged.

What is RAG?

RAG (Retrieval-Augmented Generation) is a technique that retrieves information from an external knowledge database and generates text based on it. It is primarily a concept used in Natural Language Processing (NLP), especially for the purpose of improving the accuracy of question answering (QA) and chatbots. ~ ChatGPT ~

RAG is similar to web search in that it "uses external information." The difference is that while web search generates answers based on information from the entire internet, RAG generates answers based on the content of explicitly specified data.

Table 1: Differences between RAG and Web Search

Simply put,

• RAG: Accurate in a closed world
• Web Search: Broad and new from an open world

This is how it can be summarized.

It's not a question of whether RAG or web search is superior; rather, each has its own characteristics, so they will be used differently depending on the situation. RAG applications include FAQs, knowledge search, business efficiency improvements using internal manuals, information utilization in specialized domains, and product/customer support.

Table 2: Types of Data Usable with RAG

RAG can target "almost anything that can be expressed as text."
Even images and audio can be utilized after passing through OCR or transcription.

Table 3: Environments for Utilizing RAG

Since RAG is a mechanism, you can implement it yourself, but recently, more cloud services and existing platforms make it easy to use. Broadly, they can be categorized as follows:

1Cloud AI Platforms

2Search + Generative AI Integrated Services

3No-code / Low-code Tools

While there are various services and frameworks, here I will focus on Dify, which allows building a RAG environment with no code.

Dify's Features

• GUI-centric RAG application development platform
• Create your own chat AI with no code
• Provided as a web service, with chat flow editing via a web-based GUI
• Prompt and response log management can be done via a GUI
• Has a free plan (Sandbox) (Figure 1.) *As of August 2025
  Figure 1. Dify plans and pricing

Since you can try it for free, let's start using it right away.

Creating a Chatbot App Using Dify

1. Access Dify Japanese
   Create an account from "Get Started Now."

2. Create an app > Create from scratch (Figure 2.) Figure 2. Creating an app

3. Select "Chatbot" for app type, enter the app name, and click "Create" (Figure 3.). Figure 3. Selecting the app type

4. Select the LLM to use in the red box at the top right (Figure 4.)
   *Initially, nothing is selected, so you'll need to add a model (there's a free tier for OpenAI) from the settings. Figure 4. Orchestration screen
   Here, I selected GPT-4o-mini.

5. Publish > Publish Update (Figure 5.)
   The created chatbot will be published to the web.
   You can try it out by clicking "Run App," which opens the chatbot's website in a new tab. Figure 5. App publishing

This chatbot only selected GPT-4o-mini as the language model and does not have RAG functionality added.
It simply outputs the generation result of GPT-4o-mini as is.
I tried asking about the Prime Minister in 2024 (Figure 6.).

Figure 6. Who is the Prime Minister in 2024?

The chatbot could not answer. Since GPT‑4o‑mini's knowledge cut-off date for its training data is October 2023 ², GPT‑4o‑mini cannot answer questions about a future it doesn't know.

Now, let's try building a chatbot that uses RAG.

Creating a Chatbot App Using Dify (RAG Version)

1. Prepare data for RAG
   I created a text file describing the terms of past Prime Ministers (Reiwa era only, for simplicity).

99th Prime Minister
Name: Yoshihide Suga
Term: September 16, 2020 – October 4, 2021

100th Prime Minister
Name: Fumio Kishida
Term: October 4, 2021 – November 10, 2021

101st Prime Minister
Name: Fumio Kishida
Term: November 10, 2021 – October 1, 2024

102nd Prime Minister
Name: Shigeru Ishiba
Term: October 1, 2024 – November 11, 2024

103rd Prime Minister
Name: Shigeru Ishiba
Term: November 11, 2024 – Incumbent

2. Add data to the knowledge database
   Knowledge at the top of the screen > Create Knowledge Base (Figure 7.)

3. Add Context
   Return to the chatbot's orchestration screen, and
   Context > Add (Figure 10.)

4. Operation Confirmation
   I asked "Who is the Prime Minister in 2024?" just as before.
   The chatbot is now able to answer correctly (Figure 12.).

By using Dify, you can easily create a chatbot that utilizes RAG with no code, as shown. While this example is very simple, Dify also provides means to build more advanced and complex applications. Please give it a try.