v4.6.6-search test adapt diff search mode (#685)
This commit is contained in:
parent
c766a0ed8a
commit
13b10720ac
1
.vscode/settings.json
vendored
1
.vscode/settings.json
vendored
@ -5,7 +5,6 @@
|
|||||||
"prettier.prettierPath": "./node_modules/prettier",
|
"prettier.prettierPath": "./node_modules/prettier",
|
||||||
"i18n-ally.localesPaths": [
|
"i18n-ally.localesPaths": [
|
||||||
"projects/app/public/locales",
|
"projects/app/public/locales",
|
||||||
"projects/home/public/locales"
|
|
||||||
],
|
],
|
||||||
"i18n-ally.enabledParsers": ["json"],
|
"i18n-ally.enabledParsers": ["json"],
|
||||||
"i18n-ally.keystyle": "nested",
|
"i18n-ally.keystyle": "nested",
|
||||||
|
|||||||
21
README.md
21
README.md
@ -48,19 +48,20 @@ https://github.com/labring/FastGPT/assets/15308462/7d3a38df-eb0e-4388-9250-2409b
|
|||||||
<img src="https://img.shields.io/badge/-返回顶部-7d09f1.svg" alt="#" align="right">
|
<img src="https://img.shields.io/badge/-返回顶部-7d09f1.svg" alt="#" align="right">
|
||||||
</a>
|
</a>
|
||||||
|
|
||||||
## 💡 功能
|
## 💡 RoadMap
|
||||||
|
|
||||||
`1` 强大的可视化编排,轻松构建 AI 应用
|
`1` 应用编排能力
|
||||||
- [x] 提供简易模式,无需操作编排
|
- [x] 提供简易模式,无需操作编排
|
||||||
- [x] 对话下一步指引
|
- [x] 对话下一步指引
|
||||||
- [x] 工作流编排
|
- [x] 工作流编排
|
||||||
- [x] 对话下一步指引
|
|
||||||
- [x] 源文件引用追踪
|
- [x] 源文件引用追踪
|
||||||
- [x] 模块封装,实现多级复用
|
- [x] 模块封装,实现多级复用
|
||||||
|
- [x] 混合检索 & 重排
|
||||||
|
- [ ] 自查询规划
|
||||||
- [ ] 嵌入 [Laf](https://github.com/labring/laf),实现在线编写 HTTP 模块
|
- [ ] 嵌入 [Laf](https://github.com/labring/laf),实现在线编写 HTTP 模块
|
||||||
- [ ] 混合检索 & 重排
|
- [ ] 插件封装功能
|
||||||
|
|
||||||
`2` 丰富的知识库预处理
|
`2` 知识库能力
|
||||||
- [x] 多库复用,混用
|
- [x] 多库复用,混用
|
||||||
- [x] chunk 记录修改和删除
|
- [x] chunk 记录修改和删除
|
||||||
- [x] 支持知识库单独设置向量模型
|
- [x] 支持知识库单独设置向量模型
|
||||||
@ -68,22 +69,26 @@ https://github.com/labring/FastGPT/assets/15308462/7d3a38df-eb0e-4388-9250-2409b
|
|||||||
- [x] 支持手动输入,直接分段,QA 拆分导入
|
- [x] 支持手动输入,直接分段,QA 拆分导入
|
||||||
- [x] 支持 pdf、word、txt、md 等常用文件,支持 url 读取、CSV 批量导入
|
- [x] 支持 pdf、word、txt、md 等常用文件,支持 url 读取、CSV 批量导入
|
||||||
- [ ] 支持 HTML、csv、PPT、Excel 导入
|
- [ ] 支持 HTML、csv、PPT、Excel 导入
|
||||||
|
- [ ] 支持文件阅读器
|
||||||
- [ ] 支持差异性文件同步
|
- [ ] 支持差异性文件同步
|
||||||
- [ ] 更多的数据预处理方案
|
- [ ] 更多的数据预处理方案
|
||||||
|
|
||||||
`3` 多种效果测试渠道
|
`3` 应用调试能力
|
||||||
- [x] 知识库单点搜索测试
|
- [x] 知识库单点搜索测试
|
||||||
- [x] 对话时反馈引用并可修改与删除
|
- [x] 对话时反馈引用并可修改与删除
|
||||||
- [x] 完整上下文呈现
|
- [x] 完整上下文呈现
|
||||||
- [x] 完整模块中间值呈现
|
- [x] 完整模块中间值呈现
|
||||||
|
- [ ] 高级编排 DeBug 模式
|
||||||
|
|
||||||
`4` OpenAPI
|
`4` OpenAPI 接口
|
||||||
- [x] completions 接口 (对齐 GPT 接口)
|
- [x] completions 接口 (对齐 GPT 接口)
|
||||||
- [ ] 知识库 CRUD
|
- [ ] 知识库 CRUD
|
||||||
|
- [ ] 对话 CRUD
|
||||||
|
|
||||||
`5` 运营功能
|
`5` 运营能力
|
||||||
- [x] 免登录分享窗口
|
- [x] 免登录分享窗口
|
||||||
- [x] Iframe 一键嵌入
|
- [x] Iframe 一键嵌入
|
||||||
|
- [x] 聊天窗口嵌入支持自定义 Icon,默认打开,拖拽等功能
|
||||||
- [x] 统一查阅对话记录,并对数据进行标注
|
- [x] 统一查阅对话记录,并对数据进行标注
|
||||||
|
|
||||||
<a href="#readme">
|
<a href="#readme">
|
||||||
|
|||||||
@ -307,3 +307,27 @@ weight: 708
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
```
|
```
|
||||||
|
|
||||||
|
## 特殊模型
|
||||||
|
|
||||||
|
### ReRank 接入
|
||||||
|
|
||||||
|
请使用 4.6.6-alpha 以上版本,配置文件中的 `reRankModels` 为重排模型,虽然是数组,不过目前仅有第1个生效。
|
||||||
|
|
||||||
|
1. [部署 ReRank 模型](/docs/development/custom-models/reranker/)
|
||||||
|
1. 找到 FastGPT 的配置文件中的 `reRankModels`, 4.6.6 以前是 `ReRankModels`。
|
||||||
|
2. 修改对应的值:(记得去掉注释)
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"reRankModels": [
|
||||||
|
{
|
||||||
|
"model": "bge-reranker-base", // 随意
|
||||||
|
"name": "检索重排-base", // 随意
|
||||||
|
"inputPrice": 0,
|
||||||
|
"requestUrl": "{{host}}/api/v1/rerank",
|
||||||
|
"requestAuth": "安全凭证,已自动补 Bearer"
|
||||||
|
}
|
||||||
|
]
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|||||||
64
docSite/content/docs/development/custom-models/reranker.md
Normal file
64
docSite/content/docs/development/custom-models/reranker.md
Normal file
@ -0,0 +1,64 @@
|
|||||||
|
---
|
||||||
|
title: '接入 ReRank 重排模型'
|
||||||
|
description: '接入 ReRank 重排模型'
|
||||||
|
icon: 'sort'
|
||||||
|
draft: false
|
||||||
|
toc: true
|
||||||
|
weight: 910
|
||||||
|
---
|
||||||
|
|
||||||
|
## 推荐配置
|
||||||
|
|
||||||
|
推荐配置如下:
|
||||||
|
|
||||||
|
{{< table "table-hover table-striped-columns" >}}
|
||||||
|
| 类型 | 内存 | 显存 | 硬盘空间 | 启动命令 |
|
||||||
|
|------|---------|---------|----------|--------------------------|
|
||||||
|
| base | >=4GB | >=3GB | >=8GB | python app.py |
|
||||||
|
{{< /table >}}
|
||||||
|
|
||||||
|
## 部署
|
||||||
|
|
||||||
|
### 环境要求
|
||||||
|
|
||||||
|
- Python 3.10.11
|
||||||
|
- CUDA 11.7
|
||||||
|
- 科学上网环境
|
||||||
|
|
||||||
|
### 源码部署
|
||||||
|
|
||||||
|
1. 根据上面的环境配置配置好环境,具体教程自行 GPT;
|
||||||
|
2. 下载 [python 文件](app.py)
|
||||||
|
3. 在命令行输入命令 `pip install -r requirments.txt`;
|
||||||
|
4. 按照[https://huggingface.co/BAAI/bge-reranker-base](https://huggingface.co/BAAI/bge-reranker-base)下载模型仓库到app.py同级目录
|
||||||
|
5. 添加环境变量 `export ACCESS_TOKEN=XXXXXX` 配置 token,这里的 token 只是加一层验证,防止接口被人盗用,默认值为 `ACCESS_TOKEN` ;
|
||||||
|
6. 执行命令 `python app.py`。
|
||||||
|
|
||||||
|
然后等待模型下载,直到模型加载完毕为止。如果出现报错先问 GPT。
|
||||||
|
|
||||||
|
启动成功后应该会显示如下地址:
|
||||||
|
|
||||||
|

|
||||||
|
|
||||||
|
> 这里的 `http://0.0.0.0:6006` 就是连接地址。
|
||||||
|
|
||||||
|
### docker 部署
|
||||||
|
|
||||||
|
+ 镜像名: `luanshaotong/reranker:v0.1`
|
||||||
|
+ 端口号: 6006
|
||||||
|
+ 大小:约8GB
|
||||||
|
|
||||||
|
**设置安全凭证(即oneapi中的渠道密钥)**
|
||||||
|
```
|
||||||
|
ACCESS_TOKEN=mytoken
|
||||||
|
```
|
||||||
|
|
||||||
|
**运行命令示例**
|
||||||
|
|
||||||
|
```sh
|
||||||
|
docker run -d --name reranker -p 6006:6006 -e ACCESS_TOKEN=mytoken luanshaotong/reranker:v0.1
|
||||||
|
```
|
||||||
|
|
||||||
|
## 接入 FastGPT
|
||||||
|
|
||||||
|
参考 [ReRank模型接入](/docs/development/configuration/#rerank-接入),host 变量为部署的域名。
|
||||||
@ -157,7 +157,7 @@ A2:
|
|||||||
|
|
||||||
## 搜索测试
|
## 搜索测试
|
||||||
|
|
||||||
{{< tabs tabTotal="2" >}}
|
{{< tabs tabTotal="3" >}}
|
||||||
{{< tab tabName="请求示例" >}}
|
{{< tab tabName="请求示例" >}}
|
||||||
{{< markdownify >}}
|
{{< markdownify >}}
|
||||||
|
|
||||||
@ -168,18 +168,33 @@ curl --location --request POST 'https://api.fastgpt.in/api/core/dataset/searchTe
|
|||||||
--data-raw '{
|
--data-raw '{
|
||||||
"datasetId": "知识库的ID",
|
"datasetId": "知识库的ID",
|
||||||
"text": "导演是谁",
|
"text": "导演是谁",
|
||||||
"rarank": true,
|
"limit": 5000,
|
||||||
"limit": 20
|
"similarity": 0,
|
||||||
|
"searchMode": "embedding",
|
||||||
|
"usingReRank": false
|
||||||
}'
|
}'
|
||||||
```
|
```
|
||||||
|
|
||||||
{{< /markdownify >}}
|
{{< /markdownify >}}
|
||||||
{{< /tab >}}
|
{{< /tab >}}
|
||||||
|
|
||||||
|
{{< tab tabName="参数说明" >}}
|
||||||
|
{{< markdownify >}}
|
||||||
|
|
||||||
|
- datasetId - 知识库ID
|
||||||
|
- text - 需要测试的文本
|
||||||
|
- limit - 最大 tokens 数量
|
||||||
|
- similarity - 最低相关度(0~1,可选)
|
||||||
|
- searchMode - 搜索模式:embedding | fullTextRecall | mixedRecall
|
||||||
|
- usingReRank - 使用重排
|
||||||
|
|
||||||
|
{{< /markdownify >}}
|
||||||
|
{{< /tab >}}
|
||||||
|
|
||||||
{{< tab tabName="响应示例" >}}
|
{{< tab tabName="响应示例" >}}
|
||||||
{{< markdownify >}}
|
{{< markdownify >}}
|
||||||
|
|
||||||
返回 top k 结果, limit 为预估条数,会按每条数据 800 tokens 的长度进行预估,20条也就是返回 16000 tokens 长度的数据,最多测试 30000 tokens 的数据。
|
返回 top k 结果, limit 为最大 Tokens 数量,最多 20000 tokens。
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
{
|
{
|
||||||
@ -190,15 +205,6 @@ curl --location --request POST 'https://api.fastgpt.in/api/core/dataset/searchTe
|
|||||||
"id": "65599c54a5c814fb803363cb",
|
"id": "65599c54a5c814fb803363cb",
|
||||||
"q": "你是谁",
|
"q": "你是谁",
|
||||||
"a": "我是FastGPT助手",
|
"a": "我是FastGPT助手",
|
||||||
"indexes": [
|
|
||||||
{
|
|
||||||
"defaultIndex": true,
|
|
||||||
"type": "qa",
|
|
||||||
"dataId": "3645952",
|
|
||||||
"text": "你是谁\n我是FastGPT助手",
|
|
||||||
"_id": "65599c5588271af95b019862"
|
|
||||||
}
|
|
||||||
],
|
|
||||||
"datasetId": "6554684f7f9ed18a39a4d15c",
|
"datasetId": "6554684f7f9ed18a39a4d15c",
|
||||||
"collectionId": "6556cd795e4b663e770bb66d",
|
"collectionId": "6556cd795e4b663e770bb66d",
|
||||||
"sourceName": "GBT 15104-2021 装饰单板贴面人造板.pdf",
|
"sourceName": "GBT 15104-2021 装饰单板贴面人造板.pdf",
|
||||||
|
|||||||
@ -7,12 +7,29 @@ draft: false
|
|||||||
images: []
|
images: []
|
||||||
---
|
---
|
||||||
|
|
||||||
|
## 错误排查方式
|
||||||
|
|
||||||
|
遇到问题先按下面方式排查。
|
||||||
|
|
||||||
|
1. `docker ps -a` 查看所有容器运行状态,检查是否全部 running,如有异常,尝试`docker logs 容器名`查看对应日志。
|
||||||
|
2. 不懂 docker 不要瞎改端口,只需要改`OPENAI_BASE_URL`和`CHAT_API_KEY`即可。
|
||||||
|
3. 容器都运行正常的,`docker logs 容器名` 查看报错日志
|
||||||
|
4. 无法解决时,可以找找[Issue](https://github.com/labring/FastGPT/issues),或新提 Issue,私有部署错误,务必提供详细的日志,否则很难排查。
|
||||||
|
|
||||||
|
|
||||||
## 通用问题
|
## 通用问题
|
||||||
|
|
||||||
### insufficient_user_quota user quota is not enough
|
### insufficient_user_quota user quota is not enough
|
||||||
|
|
||||||
OneAPI 账号的余额不足,默认 root 用户只有 200 刀,可以手动修改。
|
OneAPI 账号的余额不足,默认 root 用户只有 200 刀,可以手动修改。
|
||||||
|
|
||||||
|
### xxx渠道找不到
|
||||||
|
|
||||||
|
OneAPI 中没有配置该模型渠道。
|
||||||
|
|
||||||
|
### 页面中可以正常回复,API 报错
|
||||||
|
|
||||||
|
页面中是用 stream=true 模式,所以API也需要设置 stream=true 来进行测试。部分模型接口(国产居多)非 Stream 的兼容有点垃圾。
|
||||||
|
|
||||||
## Docker 部署常见问题
|
## Docker 部署常见问题
|
||||||
|
|
||||||
@ -66,11 +83,3 @@ PG 数据库没有连接上/初始化失败,可以查看日志。FastGPT 会
|
|||||||
mongo连接失败,检查
|
mongo连接失败,检查
|
||||||
1. mongo 服务有没有起来(有些 cpu 不支持 AVX,无法用 mongo5,需要换成 mongo4.x,可以dockerhub找个最新的4.x,修改镜像版本,重新运行)
|
1. mongo 服务有没有起来(有些 cpu 不支持 AVX,无法用 mongo5,需要换成 mongo4.x,可以dockerhub找个最新的4.x,修改镜像版本,重新运行)
|
||||||
2. 环境变量(账号密码,注意host和port)
|
2. 环境变量(账号密码,注意host和port)
|
||||||
|
|
||||||
|
|
||||||
### 错误排查方式
|
|
||||||
|
|
||||||
遇到问题先按下面方式排查。
|
|
||||||
|
|
||||||
1. `docker ps -a` 查看所有容器运行状态,检查是否全部 running,如有异常,尝试`docker logs 容器名`查看对应日志。
|
|
||||||
2. 不懂 docker 不要瞎改端口,只需要改`OPENAI_BASE_URL`和`CHAT_API_KEY`即可。
|
|
||||||
|
|||||||
@ -13,10 +13,9 @@ weight: 830
|
|||||||
|
|
||||||
为了减少代码重复度,我们对配置文件做了一些修改:[点击查看最新的配置文件](/docs/development/configuration/)
|
为了减少代码重复度,我们对配置文件做了一些修改:[点击查看最新的配置文件](/docs/development/configuration/)
|
||||||
|
|
||||||
|
## V4.6.6 更新说明
|
||||||
|
|
||||||
|
1. 新增 - 搜索方式:分离向量语义检索,全文检索和重排,通过 RRF 进行排序合并。
|
||||||
## V4.6.6 即将更新
|
2. 优化 - 问题分类提示词,id引导。测试国产商用 api 模型(百度阿里智谱讯飞)使用 Prompt 模式均可分类。
|
||||||
|
3. UI 优化,未来将逐步替换新的UI设计。
|
||||||
1. UI 优化,未来将逐步替换新的UI设计。
|
4. 查看 [FastGPT 2024 RoadMap](https://github.com/labring/FastGPT?tab=readme-ov-file#-%E5%9C%A8%E7%BA%BF%E4%BD%BF%E7%94%A8)
|
||||||
|
|
||||||
|
|
||||||
@ -148,5 +148,19 @@ export enum SearchScoreTypeEnum {
|
|||||||
reRank = 'reRank',
|
reRank = 'reRank',
|
||||||
rrf = 'rrf'
|
rrf = 'rrf'
|
||||||
}
|
}
|
||||||
|
export const SearchScoreTypeMap = {
|
||||||
|
[SearchScoreTypeEnum.embedding]: {
|
||||||
|
label: 'core.dataset.search.score.embedding'
|
||||||
|
},
|
||||||
|
[SearchScoreTypeEnum.fullText]: {
|
||||||
|
label: 'core.dataset.search.score.fullText'
|
||||||
|
},
|
||||||
|
[SearchScoreTypeEnum.reRank]: {
|
||||||
|
label: 'core.dataset.search.score.reRank'
|
||||||
|
},
|
||||||
|
[SearchScoreTypeEnum.rrf]: {
|
||||||
|
label: 'core.dataset.search.score.rrf'
|
||||||
|
}
|
||||||
|
};
|
||||||
|
|
||||||
export const FolderAvatarSrc = '/imgs/files/folder.svg';
|
export const FolderAvatarSrc = '/imgs/files/folder.svg';
|
||||||
|
|||||||
5
packages/global/core/dataset/type.d.ts
vendored
5
packages/global/core/dataset/type.d.ts
vendored
@ -162,7 +162,10 @@ export type DatasetFileSchema = {
|
|||||||
};
|
};
|
||||||
|
|
||||||
/* ============= search =============== */
|
/* ============= search =============== */
|
||||||
export type SearchDataResponseItemType = Omit<DatasetDataItemType, 'isOwner' | 'canWrite'> & {
|
export type SearchDataResponseItemType = Omit<
|
||||||
|
DatasetDataItemType,
|
||||||
|
'indexes' | 'isOwner' | 'canWrite'
|
||||||
|
> & {
|
||||||
score: { type: `${SearchScoreTypeEnum}`; value: number; index: number }[];
|
score: { type: `${SearchScoreTypeEnum}`; value: number; index: number }[];
|
||||||
// score: number;
|
// score: number;
|
||||||
};
|
};
|
||||||
|
|||||||
@ -27,7 +27,7 @@ export const Input_Template_History: FlowNodeInputItemType = {
|
|||||||
|
|
||||||
export const Input_Template_UserChatInput: FlowNodeInputItemType = {
|
export const Input_Template_UserChatInput: FlowNodeInputItemType = {
|
||||||
key: ModuleInputKeyEnum.userChatInput,
|
key: ModuleInputKeyEnum.userChatInput,
|
||||||
type: FlowNodeInputTypeEnum.target,
|
type: FlowNodeInputTypeEnum.hidden,
|
||||||
label: 'core.module.input.label.user question',
|
label: 'core.module.input.label.user question',
|
||||||
required: true,
|
required: true,
|
||||||
valueType: ModuleIOValueTypeEnum.string,
|
valueType: ModuleIOValueTypeEnum.string,
|
||||||
|
|||||||
@ -3,6 +3,14 @@ import { ModuleOutputKeyEnum } from '../constants';
|
|||||||
import { FlowNodeOutputTypeEnum } from '../node/constant';
|
import { FlowNodeOutputTypeEnum } from '../node/constant';
|
||||||
import { ModuleIOValueTypeEnum } from '../constants';
|
import { ModuleIOValueTypeEnum } from '../constants';
|
||||||
|
|
||||||
|
export const Output_Template_UserChatInput: FlowNodeOutputItemType = {
|
||||||
|
key: ModuleOutputKeyEnum.userChatInput,
|
||||||
|
label: 'core.module.input.label.user question',
|
||||||
|
type: FlowNodeOutputTypeEnum.hidden,
|
||||||
|
valueType: ModuleIOValueTypeEnum.string,
|
||||||
|
targets: []
|
||||||
|
};
|
||||||
|
|
||||||
export const Output_Template_Finish: FlowNodeOutputItemType = {
|
export const Output_Template_Finish: FlowNodeOutputItemType = {
|
||||||
key: ModuleOutputKeyEnum.finish,
|
key: ModuleOutputKeyEnum.finish,
|
||||||
label: 'core.module.output.label.running done',
|
label: 'core.module.output.label.running done',
|
||||||
|
|||||||
@ -16,7 +16,7 @@ import {
|
|||||||
Input_Template_UserChatInput
|
Input_Template_UserChatInput
|
||||||
} from '../input';
|
} from '../input';
|
||||||
import { chatNodeSystemPromptTip } from '../tip';
|
import { chatNodeSystemPromptTip } from '../tip';
|
||||||
import { Output_Template_Finish } from '../output';
|
import { Output_Template_Finish, Output_Template_UserChatInput } from '../output';
|
||||||
|
|
||||||
export const AiChatModule: FlowModuleTemplateType = {
|
export const AiChatModule: FlowModuleTemplateType = {
|
||||||
id: FlowNodeTypeEnum.chatNode,
|
id: FlowNodeTypeEnum.chatNode,
|
||||||
@ -131,6 +131,7 @@ export const AiChatModule: FlowModuleTemplateType = {
|
|||||||
Input_Template_UserChatInput
|
Input_Template_UserChatInput
|
||||||
],
|
],
|
||||||
outputs: [
|
outputs: [
|
||||||
|
Output_Template_UserChatInput,
|
||||||
{
|
{
|
||||||
key: ModuleOutputKeyEnum.history,
|
key: ModuleOutputKeyEnum.history,
|
||||||
label: '新的上下文',
|
label: '新的上下文',
|
||||||
|
|||||||
@ -10,6 +10,7 @@ import {
|
|||||||
Input_Template_Switch,
|
Input_Template_Switch,
|
||||||
Input_Template_UserChatInput
|
Input_Template_UserChatInput
|
||||||
} from '../input';
|
} from '../input';
|
||||||
|
import { Output_Template_UserChatInput } from '../output';
|
||||||
|
|
||||||
export const ClassifyQuestionModule: FlowModuleTemplateType = {
|
export const ClassifyQuestionModule: FlowModuleTemplateType = {
|
||||||
id: FlowNodeTypeEnum.classifyQuestion,
|
id: FlowNodeTypeEnum.classifyQuestion,
|
||||||
@ -72,6 +73,7 @@ export const ClassifyQuestionModule: FlowModuleTemplateType = {
|
|||||||
}
|
}
|
||||||
],
|
],
|
||||||
outputs: [
|
outputs: [
|
||||||
|
Output_Template_UserChatInput,
|
||||||
// custom output
|
// custom output
|
||||||
{
|
{
|
||||||
key: 'wqre',
|
key: 'wqre',
|
||||||
|
|||||||
@ -15,6 +15,7 @@ import {
|
|||||||
Input_Template_Switch,
|
Input_Template_Switch,
|
||||||
Input_Template_UserChatInput
|
Input_Template_UserChatInput
|
||||||
} from '../input';
|
} from '../input';
|
||||||
|
import { Output_Template_UserChatInput } from '../output';
|
||||||
|
|
||||||
export const AiCFR: FlowModuleTemplateType = {
|
export const AiCFR: FlowModuleTemplateType = {
|
||||||
id: FlowNodeTypeEnum.chatNode,
|
id: FlowNodeTypeEnum.chatNode,
|
||||||
@ -50,6 +51,7 @@ export const AiCFR: FlowModuleTemplateType = {
|
|||||||
Input_Template_UserChatInput
|
Input_Template_UserChatInput
|
||||||
],
|
],
|
||||||
outputs: [
|
outputs: [
|
||||||
|
Output_Template_UserChatInput,
|
||||||
{
|
{
|
||||||
key: ModuleOutputKeyEnum.text,
|
key: ModuleOutputKeyEnum.text,
|
||||||
label: 'core.module.output.label.cfr result',
|
label: 'core.module.output.label.cfr result',
|
||||||
|
|||||||
@ -11,7 +11,7 @@ import {
|
|||||||
ModuleTemplateTypeEnum
|
ModuleTemplateTypeEnum
|
||||||
} from '../../constants';
|
} from '../../constants';
|
||||||
import { Input_Template_Switch, Input_Template_UserChatInput } from '../input';
|
import { Input_Template_Switch, Input_Template_UserChatInput } from '../input';
|
||||||
import { Output_Template_Finish } from '../output';
|
import { Output_Template_Finish, Output_Template_UserChatInput } from '../output';
|
||||||
import { DatasetSearchModeEnum } from '../../../dataset/constant';
|
import { DatasetSearchModeEnum } from '../../../dataset/constant';
|
||||||
|
|
||||||
export const DatasetSearchModule: FlowModuleTemplateType = {
|
export const DatasetSearchModule: FlowModuleTemplateType = {
|
||||||
@ -90,6 +90,7 @@ export const DatasetSearchModule: FlowModuleTemplateType = {
|
|||||||
Input_Template_UserChatInput
|
Input_Template_UserChatInput
|
||||||
],
|
],
|
||||||
outputs: [
|
outputs: [
|
||||||
|
Output_Template_UserChatInput,
|
||||||
{
|
{
|
||||||
key: ModuleOutputKeyEnum.datasetIsEmpty,
|
key: ModuleOutputKeyEnum.datasetIsEmpty,
|
||||||
label: '搜索结果为空',
|
label: '搜索结果为空',
|
||||||
|
|||||||
@ -15,7 +15,7 @@ import {
|
|||||||
Input_Template_Switch,
|
Input_Template_Switch,
|
||||||
Input_Template_UserChatInput
|
Input_Template_UserChatInput
|
||||||
} from '../input';
|
} from '../input';
|
||||||
import { Output_Template_Finish } from '../output';
|
import { Output_Template_Finish, Output_Template_UserChatInput } from '../output';
|
||||||
|
|
||||||
export const RunAppModule: FlowModuleTemplateType = {
|
export const RunAppModule: FlowModuleTemplateType = {
|
||||||
id: FlowNodeTypeEnum.runApp,
|
id: FlowNodeTypeEnum.runApp,
|
||||||
@ -41,6 +41,7 @@ export const RunAppModule: FlowModuleTemplateType = {
|
|||||||
Input_Template_UserChatInput
|
Input_Template_UserChatInput
|
||||||
],
|
],
|
||||||
outputs: [
|
outputs: [
|
||||||
|
Output_Template_UserChatInput,
|
||||||
{
|
{
|
||||||
key: ModuleOutputKeyEnum.history,
|
key: ModuleOutputKeyEnum.history,
|
||||||
label: '新的上下文',
|
label: '新的上下文',
|
||||||
|
|||||||
@ -24,10 +24,9 @@ export async function initPg() {
|
|||||||
);
|
);
|
||||||
`);
|
`);
|
||||||
|
|
||||||
await PgClient.query(`
|
await PgClient.query(
|
||||||
CREATE INDEX CONCURRENTLY IF NOT EXISTS vector_index ON ${PgDatasetTableName}
|
`CREATE INDEX CONCURRENTLY IF NOT EXISTS vector_index ON ${PgDatasetTableName} USING hnsw (vector vector_ip_ops) WITH (m = 32, ef_construction = 64);`
|
||||||
USING hnsw (vector vector_ip_ops) WITH (m = 32, ef_construction = 64);
|
);
|
||||||
`);
|
|
||||||
|
|
||||||
console.log('init pg successful');
|
console.log('init pg successful');
|
||||||
} catch (error) {
|
} catch (error) {
|
||||||
|
|||||||
@ -80,8 +80,8 @@ const ChatItemSchema = new Schema({
|
|||||||
});
|
});
|
||||||
|
|
||||||
try {
|
try {
|
||||||
|
ChatItemSchema.index({ dataId: -1 });
|
||||||
ChatItemSchema.index({ time: -1 });
|
ChatItemSchema.index({ time: -1 });
|
||||||
ChatItemSchema.index({ userId: 1 });
|
|
||||||
ChatItemSchema.index({ appId: 1 });
|
ChatItemSchema.index({ appId: 1 });
|
||||||
ChatItemSchema.index({ chatId: 1 });
|
ChatItemSchema.index({ chatId: 1 });
|
||||||
ChatItemSchema.index({ userGoodFeedback: 1 });
|
ChatItemSchema.index({ userGoodFeedback: 1 });
|
||||||
|
|||||||
@ -86,6 +86,8 @@ export const getSameRawTextCollection = async ({
|
|||||||
datasetId: string;
|
datasetId: string;
|
||||||
hashRawText?: string;
|
hashRawText?: string;
|
||||||
}) => {
|
}) => {
|
||||||
|
if (!hashRawText) return undefined;
|
||||||
|
|
||||||
const collection = await MongoDatasetCollection.findOne({
|
const collection = await MongoDatasetCollection.findOne({
|
||||||
datasetId,
|
datasetId,
|
||||||
hashRawText
|
hashRawText
|
||||||
|
|||||||
@ -1,4 +1,4 @@
|
|||||||
### Fast GPT V4.6.5
|
### Fast GPT V4.6.6
|
||||||
|
|
||||||
1. 新增 - [问题补全模块](https://doc.fastgpt.in/docs/workflow/modules/coreferenceresolution/)
|
1. 新增 - [问题补全模块](https://doc.fastgpt.in/docs/workflow/modules/coreferenceresolution/)
|
||||||
2. 新增 - [文本编辑模块](https://doc.fastgpt.in/docs/workflow/modules/text_editor/)
|
2. 新增 - [文本编辑模块](https://doc.fastgpt.in/docs/workflow/modules/text_editor/)
|
||||||
|
|||||||
@ -296,6 +296,7 @@
|
|||||||
"Select Image": "Select Image",
|
"Select Image": "Select Image",
|
||||||
"Send Message": "Send Message",
|
"Send Message": "Send Message",
|
||||||
"Speaking": "I'm listening...",
|
"Speaking": "I'm listening...",
|
||||||
|
"Start Chat": "Start Chat",
|
||||||
"Stop Speak": "Stop Speak",
|
"Stop Speak": "Stop Speak",
|
||||||
"Type a message": "Input problem",
|
"Type a message": "Input problem",
|
||||||
"error": {
|
"error": {
|
||||||
@ -379,6 +380,7 @@
|
|||||||
"Name": "Name",
|
"Name": "Name",
|
||||||
"Quote Length": "Quote Length",
|
"Quote Length": "Quote Length",
|
||||||
"Read Dataset": "Read Dataset",
|
"Read Dataset": "Read Dataset",
|
||||||
|
"Search score tip": "{{scoreText}}Here are the rankings and scores:\n----\n{{detailScore}}",
|
||||||
"Set Empty Result Tip": ",Response empty text",
|
"Set Empty Result Tip": ",Response empty text",
|
||||||
"Set Website Config": "Configuring Website",
|
"Set Website Config": "Configuring Website",
|
||||||
"Similarity": "Similarity",
|
"Similarity": "Similarity",
|
||||||
@ -435,6 +437,7 @@
|
|||||||
"Too Long": "Content is too long",
|
"Too Long": "Content is too long",
|
||||||
"Total Amount": "{{total}} Chunks",
|
"Total Amount": "{{total}} Chunks",
|
||||||
"data is deleted": "Data is deleted",
|
"data is deleted": "Data is deleted",
|
||||||
|
"get data error": "Get data error",
|
||||||
"id": "Data ID"
|
"id": "Data ID"
|
||||||
},
|
},
|
||||||
"error": {
|
"error": {
|
||||||
@ -482,6 +485,7 @@
|
|||||||
"link": "Link",
|
"link": "Link",
|
||||||
"search": {
|
"search": {
|
||||||
"Dataset Search Params": "Dataset Search Params",
|
"Dataset Search Params": "Dataset Search Params",
|
||||||
|
"Embedding score": "Embedding score",
|
||||||
"Empty result response": "Empty Response",
|
"Empty result response": "Empty Response",
|
||||||
"Empty result response Tips": "If you fill in the content, if no suitable content is found, you will directly reply to the content.",
|
"Empty result response Tips": "If you fill in the content, if no suitable content is found, you will directly reply to the content.",
|
||||||
"Max Tokens": "Max Tokens",
|
"Max Tokens": "Max Tokens",
|
||||||
@ -489,8 +493,14 @@
|
|||||||
"Min Similarity": "Min Similarity",
|
"Min Similarity": "Min Similarity",
|
||||||
"Min Similarity Tips": "The similarity of different index models is different, please use the search test to select the appropriate value",
|
"Min Similarity Tips": "The similarity of different index models is different, please use the search test to select the appropriate value",
|
||||||
"Params Setting": "Params Setting",
|
"Params Setting": "Params Setting",
|
||||||
|
"Rank": "Rank",
|
||||||
|
"Rank Tip": "Ranking in all data",
|
||||||
"ReRank": "ReRank",
|
"ReRank": "ReRank",
|
||||||
"ReRank desc": "The rearrangement model is used for secondary ranking to enhance the overall ranking.",
|
"ReRank desc": "The rearrangement model is used for secondary ranking to enhance the overall ranking.",
|
||||||
|
"Read score": "Read score",
|
||||||
|
"Rerank score": "ReRank score",
|
||||||
|
"Score": "Score",
|
||||||
|
"Search type": "Type",
|
||||||
"Top K": "Top K",
|
"Top K": "Top K",
|
||||||
"mode": {
|
"mode": {
|
||||||
"embedding": "Vector search",
|
"embedding": "Vector search",
|
||||||
@ -500,6 +510,12 @@
|
|||||||
"mixedRecall": "Mixedrecall",
|
"mixedRecall": "Mixedrecall",
|
||||||
"mixedRecall desc": "Returns the combined results of vector and full-text searches, sorted using the RRF algorithm."
|
"mixedRecall desc": "Returns the combined results of vector and full-text searches, sorted using the RRF algorithm."
|
||||||
},
|
},
|
||||||
|
"score": {
|
||||||
|
"embedding": "Embedding",
|
||||||
|
"fullText": "Full text",
|
||||||
|
"reRank": "ReRank",
|
||||||
|
"rrf": "RRF Merge"
|
||||||
|
},
|
||||||
"search mode": "Search Mode"
|
"search mode": "Search Mode"
|
||||||
},
|
},
|
||||||
"status": {
|
"status": {
|
||||||
@ -507,11 +523,15 @@
|
|||||||
"syncing": "Syncing"
|
"syncing": "Syncing"
|
||||||
},
|
},
|
||||||
"test": {
|
"test": {
|
||||||
|
"Batch test": "Batch test",
|
||||||
|
"Batch test Placeholder": "Select one csv file",
|
||||||
"Search Test": "Search Test",
|
"Search Test": "Search Test",
|
||||||
"Test": "Start",
|
"Test": "Start",
|
||||||
|
"Test File": "",
|
||||||
"Test Result": "Results",
|
"Test Result": "Results",
|
||||||
"Test Text": "Text",
|
"Test Text": "Text",
|
||||||
"Test Text Placeholder": "Enter the text you want to test",
|
"Test Text Placeholder": "Enter the text you want to test",
|
||||||
|
"Test params": "Search Params",
|
||||||
"delete test history": "Delete the test result",
|
"delete test history": "Delete the test result",
|
||||||
"test history": "Test History",
|
"test history": "Test History",
|
||||||
"test result placeholder": "The test results will be presented here",
|
"test result placeholder": "The test results will be presented here",
|
||||||
|
|||||||
@ -296,6 +296,7 @@
|
|||||||
"Select Image": "选择图片",
|
"Select Image": "选择图片",
|
||||||
"Send Message": "发送",
|
"Send Message": "发送",
|
||||||
"Speaking": "我在听,请说...",
|
"Speaking": "我在听,请说...",
|
||||||
|
"Start Chat": "开始对话",
|
||||||
"Stop Speak": "停止录音",
|
"Stop Speak": "停止录音",
|
||||||
"Type a message": "输入问题",
|
"Type a message": "输入问题",
|
||||||
"error": {
|
"error": {
|
||||||
@ -379,6 +380,7 @@
|
|||||||
"Name": "知识库名称",
|
"Name": "知识库名称",
|
||||||
"Quote Length": "引用内容长度",
|
"Quote Length": "引用内容长度",
|
||||||
"Read Dataset": "查看知识库详情",
|
"Read Dataset": "查看知识库详情",
|
||||||
|
"Search score tip": "{{scoreText}}下面是详细排名和得分情况:\n----\n{{detailScore}}",
|
||||||
"Set Empty Result Tip": ",未搜索到内容时回复指定内容",
|
"Set Empty Result Tip": ",未搜索到内容时回复指定内容",
|
||||||
"Set Website Config": "开始配置网站信息",
|
"Set Website Config": "开始配置网站信息",
|
||||||
"Similarity": "相关度",
|
"Similarity": "相关度",
|
||||||
@ -435,6 +437,7 @@
|
|||||||
"Too Long": "总长度超长了",
|
"Too Long": "总长度超长了",
|
||||||
"Total Amount": "{{total}} 组",
|
"Total Amount": "{{total}} 组",
|
||||||
"data is deleted": "该数据已被删除",
|
"data is deleted": "该数据已被删除",
|
||||||
|
"get data error": "获取数据异常",
|
||||||
"id": "数据ID"
|
"id": "数据ID"
|
||||||
},
|
},
|
||||||
"error": {
|
"error": {
|
||||||
@ -482,6 +485,7 @@
|
|||||||
"link": "链接",
|
"link": "链接",
|
||||||
"search": {
|
"search": {
|
||||||
"Dataset Search Params": "搜索参数",
|
"Dataset Search Params": "搜索参数",
|
||||||
|
"Embedding score": "语意检索得分",
|
||||||
"Empty result response": "空搜索回复",
|
"Empty result response": "空搜索回复",
|
||||||
"Empty result response Tips": "若填写该内容,没有搜索到合适内容时,将直接回复填写的内容。",
|
"Empty result response Tips": "若填写该内容,没有搜索到合适内容时,将直接回复填写的内容。",
|
||||||
"Max Tokens": "引用上限",
|
"Max Tokens": "引用上限",
|
||||||
@ -489,8 +493,14 @@
|
|||||||
"Min Similarity": "最低相关度",
|
"Min Similarity": "最低相关度",
|
||||||
"Min Similarity Tips": "不同索引模型的相关度有区别,请通过搜索测试来选择合适的数值,使用 ReRank 时,相关度可能会很低。",
|
"Min Similarity Tips": "不同索引模型的相关度有区别,请通过搜索测试来选择合适的数值,使用 ReRank 时,相关度可能会很低。",
|
||||||
"Params Setting": "搜索参数设置",
|
"Params Setting": "搜索参数设置",
|
||||||
|
"Rank": "排名",
|
||||||
|
"Rank Tip": "在所有数据中的排名",
|
||||||
"ReRank": "结果重排",
|
"ReRank": "结果重排",
|
||||||
"ReRank desc": "使用重排模型来进行二次排序,可增强综合排名。",
|
"ReRank desc": "使用重排模型来进行二次排序,可增强综合排名。",
|
||||||
|
"Read score": "查看得分",
|
||||||
|
"Rerank score": "结果重排得分",
|
||||||
|
"Score": "得分",
|
||||||
|
"Search type": "类型",
|
||||||
"Top K": "单次搜索上限",
|
"Top K": "单次搜索上限",
|
||||||
"mode": {
|
"mode": {
|
||||||
"embedding": "语义检索",
|
"embedding": "语义检索",
|
||||||
@ -500,6 +510,12 @@
|
|||||||
"mixedRecall": "混合检索",
|
"mixedRecall": "混合检索",
|
||||||
"mixedRecall desc": "使用向量检索与全文检索的综合结果返回,使用RRF算法进行排序。"
|
"mixedRecall desc": "使用向量检索与全文检索的综合结果返回,使用RRF算法进行排序。"
|
||||||
},
|
},
|
||||||
|
"score": {
|
||||||
|
"embedding": "语义检索",
|
||||||
|
"fullText": "全文检索",
|
||||||
|
"reRank": "结果重排",
|
||||||
|
"rrf": "RRF 合并"
|
||||||
|
},
|
||||||
"search mode": "搜索模式"
|
"search mode": "搜索模式"
|
||||||
},
|
},
|
||||||
"status": {
|
"status": {
|
||||||
@ -507,11 +523,15 @@
|
|||||||
"syncing": "同步中"
|
"syncing": "同步中"
|
||||||
},
|
},
|
||||||
"test": {
|
"test": {
|
||||||
|
"Batch test": "批量测试",
|
||||||
|
"Batch test Placeholder": "选择一个 Csv 文件",
|
||||||
"Search Test": "搜索测试",
|
"Search Test": "搜索测试",
|
||||||
"Test": "测试",
|
"Test": "测试",
|
||||||
|
"Test File": "批量测试",
|
||||||
"Test Result": "测试结果",
|
"Test Result": "测试结果",
|
||||||
"Test Text": "测试文本",
|
"Test Text": "单个文本测试",
|
||||||
"Test Text Placeholder": "输入需要测试的文本",
|
"Test Text Placeholder": "输入需要测试的文本",
|
||||||
|
"Test params": "测试参数",
|
||||||
"delete test history": "删除该测试结果",
|
"delete test history": "删除该测试结果",
|
||||||
"test history": "测试历史",
|
"test history": "测试历史",
|
||||||
"test result placeholder": "测试结果将在这里展示",
|
"test result placeholder": "测试结果将在这里展示",
|
||||||
@ -847,7 +867,7 @@
|
|||||||
"Response Detail": "返回详情",
|
"Response Detail": "返回详情",
|
||||||
"Response Detail tips": "是否需要返回详情(引用内容,调用时间等,不会返回预设提示词和完整上下文)",
|
"Response Detail tips": "是否需要返回详情(引用内容,调用时间等,不会返回预设提示词和完整上下文)",
|
||||||
"token auth": "身份验证",
|
"token auth": "身份验证",
|
||||||
"token auth Tips": "身份校验服务器地址,如填写该值,每次对话前都会想指定服务器发送一个请求,进行身份校验",
|
"token auth Tips": "身份校验服务器地址,如填写该值,每次对话前都会向指定服务器发送一个请求,进行身份校验",
|
||||||
"token auth use cases": "查看身份验证使用说明"
|
"token auth use cases": "查看身份验证使用说明"
|
||||||
},
|
},
|
||||||
"permission": {
|
"permission": {
|
||||||
|
|||||||
@ -32,7 +32,7 @@ const ContextModal = ({
|
|||||||
<Box
|
<Box
|
||||||
key={i}
|
key={i}
|
||||||
p={2}
|
p={2}
|
||||||
borderRadius={'lg'}
|
borderRadius={'md'}
|
||||||
border={theme.borders.base}
|
border={theme.borders.base}
|
||||||
_notLast={{ mb: 2 }}
|
_notLast={{ mb: 2 }}
|
||||||
position={'relative'}
|
position={'relative'}
|
||||||
|
|||||||
@ -9,12 +9,16 @@ const FeedbackModal = ({
|
|||||||
appId,
|
appId,
|
||||||
chatId,
|
chatId,
|
||||||
chatItemId,
|
chatItemId,
|
||||||
|
shareId,
|
||||||
|
outLinkUid,
|
||||||
onSuccess,
|
onSuccess,
|
||||||
onClose
|
onClose
|
||||||
}: {
|
}: {
|
||||||
appId: string;
|
appId: string;
|
||||||
chatId: string;
|
chatId: string;
|
||||||
chatItemId: string;
|
chatItemId: string;
|
||||||
|
shareId?: string;
|
||||||
|
outLinkUid?: string;
|
||||||
onSuccess: (e: string) => void;
|
onSuccess: (e: string) => void;
|
||||||
onClose: () => void;
|
onClose: () => void;
|
||||||
}) => {
|
}) => {
|
||||||
@ -28,6 +32,8 @@ const FeedbackModal = ({
|
|||||||
appId,
|
appId,
|
||||||
chatId,
|
chatId,
|
||||||
chatItemId,
|
chatItemId,
|
||||||
|
shareId,
|
||||||
|
outLinkUid,
|
||||||
userBadFeedback: val
|
userBadFeedback: val
|
||||||
});
|
});
|
||||||
},
|
},
|
||||||
|
|||||||
@ -1,31 +1,38 @@
|
|||||||
import React, { useCallback, useState } from 'react';
|
import React, { useMemo, useState } from 'react';
|
||||||
import { ModalBody, Box, useTheme, Flex, Progress, Link } from '@chakra-ui/react';
|
import { ModalBody, Box, useTheme } from '@chakra-ui/react';
|
||||||
import { getDatasetDataItemById } from '@/web/core/dataset/api';
|
|
||||||
import { useLoading } from '@/web/common/hooks/useLoading';
|
|
||||||
import { useToast } from '@/web/common/hooks/useToast';
|
|
||||||
import { getErrText } from '@fastgpt/global/common/error/utils';
|
|
||||||
import MyIcon from '@/components/Icon';
|
|
||||||
import InputDataModal, {
|
|
||||||
RawSourceText,
|
|
||||||
type InputDataType
|
|
||||||
} from '@/pages/dataset/detail/components/InputDataModal';
|
|
||||||
import MyModal from '../MyModal';
|
import MyModal from '../MyModal';
|
||||||
import { useTranslation } from 'next-i18next';
|
import { useTranslation } from 'next-i18next';
|
||||||
import type { SearchDataResponseItemType } from '@fastgpt/global/core/dataset/type';
|
import type { SearchDataResponseItemType } from '@fastgpt/global/core/dataset/type';
|
||||||
import MyTooltip from '../MyTooltip';
|
import QuoteItem from '../core/dataset/QuoteItem';
|
||||||
import NextLink from 'next/link';
|
import { RawSourceText } from '@/pages/dataset/detail/components/InputDataModal';
|
||||||
import { useSystemStore } from '@/web/common/system/useSystemStore';
|
|
||||||
|
|
||||||
const QuoteModal = ({
|
const QuoteModal = ({
|
||||||
rawSearch = [],
|
rawSearch = [],
|
||||||
onClose,
|
onClose,
|
||||||
isShare
|
isShare,
|
||||||
|
metadata
|
||||||
}: {
|
}: {
|
||||||
rawSearch: SearchDataResponseItemType[];
|
rawSearch: SearchDataResponseItemType[];
|
||||||
onClose: () => void;
|
onClose: () => void;
|
||||||
isShare: boolean;
|
isShare: boolean;
|
||||||
|
metadata?: {
|
||||||
|
collectionId: string;
|
||||||
|
sourceId?: string;
|
||||||
|
sourceName: string;
|
||||||
|
};
|
||||||
}) => {
|
}) => {
|
||||||
const { t } = useTranslation();
|
const { t } = useTranslation();
|
||||||
|
const filterResults = useMemo(
|
||||||
|
() =>
|
||||||
|
metadata
|
||||||
|
? rawSearch.filter(
|
||||||
|
(item) =>
|
||||||
|
item.collectionId === metadata.collectionId && item.sourceId === metadata.sourceId
|
||||||
|
)
|
||||||
|
: rawSearch,
|
||||||
|
[metadata, rawSearch]
|
||||||
|
);
|
||||||
|
|
||||||
return (
|
return (
|
||||||
<>
|
<>
|
||||||
@ -35,18 +42,22 @@ const QuoteModal = ({
|
|||||||
h={['90vh', '80vh']}
|
h={['90vh', '80vh']}
|
||||||
isCentered
|
isCentered
|
||||||
minW={['90vw', '600px']}
|
minW={['90vw', '600px']}
|
||||||
iconSrc="/imgs/modal/quote.svg"
|
iconSrc={!!metadata ? undefined : '/imgs/modal/quote.svg'}
|
||||||
title={
|
title={
|
||||||
<Box>
|
<Box>
|
||||||
{t('core.chat.Quote Amount', { amount: rawSearch.length })}
|
{metadata ? (
|
||||||
<Box fontSize={'sm'} color={'myGray.500'} fontWeight={'normal'}>
|
<RawSourceText {...metadata} canView={false} />
|
||||||
|
) : (
|
||||||
|
<>{t('core.chat.Quote Amount', { amount: rawSearch.length })}</>
|
||||||
|
)}
|
||||||
|
<Box fontSize={'xs'} color={'myGray.500'} fontWeight={'normal'}>
|
||||||
{t('core.chat.quote.Quote Tip')}
|
{t('core.chat.quote.Quote Tip')}
|
||||||
</Box>
|
</Box>
|
||||||
</Box>
|
</Box>
|
||||||
}
|
}
|
||||||
>
|
>
|
||||||
<ModalBody whiteSpace={'pre-wrap'} textAlign={'justify'} wordBreak={'break-all'}>
|
<ModalBody>
|
||||||
<QuoteList rawSearch={rawSearch} isShare={isShare} />
|
<QuoteList rawSearch={filterResults} isShare={isShare} />
|
||||||
</ModalBody>
|
</ModalBody>
|
||||||
</MyModal>
|
</MyModal>
|
||||||
</>
|
</>
|
||||||
@ -62,38 +73,7 @@ export const QuoteList = React.memo(function QuoteList({
|
|||||||
rawSearch: SearchDataResponseItemType[];
|
rawSearch: SearchDataResponseItemType[];
|
||||||
isShare: boolean;
|
isShare: boolean;
|
||||||
}) {
|
}) {
|
||||||
const { t } = useTranslation();
|
|
||||||
const { isPc } = useSystemStore();
|
|
||||||
const theme = useTheme();
|
const theme = useTheme();
|
||||||
const { toast } = useToast();
|
|
||||||
const { setIsLoading, Loading } = useLoading();
|
|
||||||
const [editInputData, setEditInputData] = useState<InputDataType & { collectionId: string }>();
|
|
||||||
|
|
||||||
/**
|
|
||||||
* click edit, get new DataItem
|
|
||||||
*/
|
|
||||||
const onclickEdit = useCallback(
|
|
||||||
async (item: InputDataType) => {
|
|
||||||
if (!item.id) return;
|
|
||||||
try {
|
|
||||||
setIsLoading(true);
|
|
||||||
const data = await getDatasetDataItemById(item.id);
|
|
||||||
|
|
||||||
if (!data) {
|
|
||||||
throw new Error('该数据已被删除');
|
|
||||||
}
|
|
||||||
|
|
||||||
setEditInputData(data);
|
|
||||||
} catch (err) {
|
|
||||||
toast({
|
|
||||||
status: 'warning',
|
|
||||||
title: getErrText(err)
|
|
||||||
});
|
|
||||||
}
|
|
||||||
setIsLoading(false);
|
|
||||||
},
|
|
||||||
[setIsLoading, toast]
|
|
||||||
);
|
|
||||||
|
|
||||||
return (
|
return (
|
||||||
<>
|
<>
|
||||||
@ -102,113 +82,15 @@ export const QuoteList = React.memo(function QuoteList({
|
|||||||
key={i}
|
key={i}
|
||||||
flex={'1 0 0'}
|
flex={'1 0 0'}
|
||||||
p={2}
|
p={2}
|
||||||
borderRadius={'lg'}
|
borderRadius={'sm'}
|
||||||
border={theme.borders.base}
|
border={theme.borders.base}
|
||||||
_notLast={{ mb: 2 }}
|
_notLast={{ mb: 2 }}
|
||||||
position={'relative'}
|
|
||||||
overflow={'hidden'}
|
|
||||||
_hover={{ '& .hover-data': { display: 'flex' } }}
|
_hover={{ '& .hover-data': { display: 'flex' } }}
|
||||||
bg={i % 2 === 0 ? 'white' : 'myWhite.500'}
|
bg={i % 2 === 0 ? 'white' : 'myWhite.500'}
|
||||||
>
|
>
|
||||||
<Flex alignItems={'flex-end'} mb={3} fontSize={'sm'}>
|
<QuoteItem quoteItem={item} canViewSource={!isShare} linkToDataset={!isShare} />
|
||||||
<RawSourceText
|
|
||||||
fontWeight={'bold'}
|
|
||||||
color={'black'}
|
|
||||||
sourceName={item.sourceName}
|
|
||||||
sourceId={item.sourceId}
|
|
||||||
canView={!isShare}
|
|
||||||
/>
|
|
||||||
<Box flex={1} />
|
|
||||||
{!isShare && (
|
|
||||||
<Link
|
|
||||||
as={NextLink}
|
|
||||||
className="hover-data"
|
|
||||||
display={'none'}
|
|
||||||
alignItems={'center'}
|
|
||||||
color={'primary.500'}
|
|
||||||
href={`/dataset/detail?datasetId=${item.datasetId}¤tTab=dataCard&collectionId=${item.collectionId}`}
|
|
||||||
>
|
|
||||||
{t('core.dataset.Go Dataset')}
|
|
||||||
<MyIcon name={'common/rightArrowLight'} w={'10px'} />
|
|
||||||
</Link>
|
|
||||||
)}
|
|
||||||
</Flex>
|
|
||||||
|
|
||||||
<Box color={'black'}>{item.q}</Box>
|
|
||||||
<Box color={'myGray.600'}>{item.a}</Box>
|
|
||||||
{!isShare && (
|
|
||||||
<Flex alignItems={'center'} fontSize={'sm'} mt={3} gap={4} color={'myGray.500'}>
|
|
||||||
{isPc && (
|
|
||||||
<MyTooltip label={t('core.dataset.data.id')}>
|
|
||||||
<Flex border={theme.borders.base} py={'1px'} px={3} borderRadius={'3px'}>
|
|
||||||
# {item.id}
|
|
||||||
</Flex>
|
|
||||||
</MyTooltip>
|
|
||||||
)}
|
|
||||||
<MyTooltip label={t('core.dataset.Quote Length')}>
|
|
||||||
<Flex alignItems={'center'}>
|
|
||||||
<MyIcon name="common/text/t" w={'14px'} mr={1} color={'myGray.500'} />
|
|
||||||
{item.q.length + (item.a?.length || 0)}
|
|
||||||
</Flex>
|
|
||||||
</MyTooltip>
|
|
||||||
{/* {!isShare && item.score && (
|
|
||||||
<MyTooltip label={t('core.dataset.Similarity')}>
|
|
||||||
<Flex alignItems={'center'}>
|
|
||||||
<MyIcon name={'kbTest'} w={'12px'} />
|
|
||||||
<Progress
|
|
||||||
mx={2}
|
|
||||||
w={['60px', '90px']}
|
|
||||||
value={item.score * 100}
|
|
||||||
size="sm"
|
|
||||||
borderRadius={'20px'}
|
|
||||||
colorScheme="myGray"
|
|
||||||
border={theme.borders.base}
|
|
||||||
/>
|
|
||||||
<Box>{item.score.toFixed(4)}</Box>
|
|
||||||
</Flex>
|
|
||||||
</MyTooltip>
|
|
||||||
)} */}
|
|
||||||
<Box flex={1} />
|
|
||||||
{item.id && (
|
|
||||||
<MyTooltip label={t('core.dataset.data.Edit')}>
|
|
||||||
<Box
|
|
||||||
bg={'rgba(255,255,255,0.9)'}
|
|
||||||
alignItems={'center'}
|
|
||||||
justifyContent={'center'}
|
|
||||||
boxShadow={'-10px 0 10px rgba(255,255,255,1)'}
|
|
||||||
>
|
|
||||||
<MyIcon
|
|
||||||
name={'edit'}
|
|
||||||
w={['16px', '18px']}
|
|
||||||
h={['16px', '18px']}
|
|
||||||
cursor={'pointer'}
|
|
||||||
color={'myGray.600'}
|
|
||||||
_hover={{
|
|
||||||
color: 'primary.600'
|
|
||||||
}}
|
|
||||||
onClick={() => onclickEdit(item)}
|
|
||||||
/>
|
|
||||||
</Box>
|
|
||||||
</MyTooltip>
|
|
||||||
)}
|
|
||||||
</Flex>
|
|
||||||
)}
|
|
||||||
</Box>
|
</Box>
|
||||||
))}
|
))}
|
||||||
{editInputData && editInputData.id && (
|
|
||||||
<InputDataModal
|
|
||||||
onClose={() => setEditInputData(undefined)}
|
|
||||||
onSuccess={() => {
|
|
||||||
console.log('更新引用成功');
|
|
||||||
}}
|
|
||||||
onDelete={() => {
|
|
||||||
console.log('删除引用成功');
|
|
||||||
}}
|
|
||||||
defaultValue={editInputData}
|
|
||||||
collectionId={editInputData.collectionId}
|
|
||||||
/>
|
|
||||||
)}
|
|
||||||
<Loading fixed={false} />
|
|
||||||
</>
|
</>
|
||||||
);
|
);
|
||||||
});
|
});
|
||||||
|
|||||||
@ -11,8 +11,6 @@ import MyTooltip from '../MyTooltip';
|
|||||||
import { FlowNodeTypeEnum } from '@fastgpt/global/core/module/node/constant';
|
import { FlowNodeTypeEnum } from '@fastgpt/global/core/module/node/constant';
|
||||||
import { getSourceNameIcon } from '@fastgpt/global/core/dataset/utils';
|
import { getSourceNameIcon } from '@fastgpt/global/core/dataset/utils';
|
||||||
import ChatBoxDivider from '@/components/core/chat/Divider';
|
import ChatBoxDivider from '@/components/core/chat/Divider';
|
||||||
import MyIcon from '../Icon';
|
|
||||||
import { getFileAndOpen } from '@/web/core/dataset/utils';
|
|
||||||
import { strIsLink } from '@fastgpt/global/common/string/tools';
|
import { strIsLink } from '@fastgpt/global/common/string/tools';
|
||||||
|
|
||||||
const QuoteModal = dynamic(() => import('./QuoteModal'), { ssr: false });
|
const QuoteModal = dynamic(() => import('./QuoteModal'), { ssr: false });
|
||||||
@ -29,7 +27,14 @@ const ResponseTags = ({
|
|||||||
const theme = useTheme();
|
const theme = useTheme();
|
||||||
const { isPc } = useSystemStore();
|
const { isPc } = useSystemStore();
|
||||||
const { t } = useTranslation();
|
const { t } = useTranslation();
|
||||||
const [quoteModalData, setQuoteModalData] = useState<SearchDataResponseItemType[]>();
|
const [quoteModalData, setQuoteModalData] = useState<{
|
||||||
|
rawSearch: SearchDataResponseItemType[];
|
||||||
|
metadata?: {
|
||||||
|
collectionId: string;
|
||||||
|
sourceId?: string;
|
||||||
|
sourceName: string;
|
||||||
|
};
|
||||||
|
}>();
|
||||||
const [contextModalData, setContextModalData] = useState<ChatItemType[]>();
|
const [contextModalData, setContextModalData] = useState<ChatItemType[]>();
|
||||||
const {
|
const {
|
||||||
isOpen: isOpenWholeModal,
|
isOpen: isOpenWholeModal,
|
||||||
@ -52,8 +57,8 @@ const ResponseTags = ({
|
|||||||
.filter(Boolean) as SearchDataResponseItemType[];
|
.filter(Boolean) as SearchDataResponseItemType[];
|
||||||
const sourceList = quoteList.reduce(
|
const sourceList = quoteList.reduce(
|
||||||
(acc: Record<string, SearchDataResponseItemType[]>, cur) => {
|
(acc: Record<string, SearchDataResponseItemType[]>, cur) => {
|
||||||
if (!acc[cur.sourceName]) {
|
if (!acc[cur.collectionId]) {
|
||||||
acc[cur.sourceName] = [cur];
|
acc[cur.collectionId] = [cur];
|
||||||
}
|
}
|
||||||
return acc;
|
return acc;
|
||||||
},
|
},
|
||||||
@ -70,7 +75,8 @@ const ResponseTags = ({
|
|||||||
sourceName: item.sourceName,
|
sourceName: item.sourceName,
|
||||||
sourceId: item.sourceId,
|
sourceId: item.sourceId,
|
||||||
icon: getSourceNameIcon({ sourceId: item.sourceId, sourceName: item.sourceName }),
|
icon: getSourceNameIcon({ sourceId: item.sourceId, sourceName: item.sourceName }),
|
||||||
canReadQuote: !isShare || strIsLink(item.sourceId)
|
canReadQuote: !isShare || strIsLink(item.sourceId),
|
||||||
|
collectionId: item.collectionId
|
||||||
})),
|
})),
|
||||||
historyPreview: chatData?.historyPreview,
|
historyPreview: chatData?.historyPreview,
|
||||||
runningTime: +responseData.reduce((sum, item) => sum + (item.runningTime || 0), 0).toFixed(2)
|
runningTime: +responseData.reduce((sum, item) => sum + (item.runningTime || 0), 0).toFixed(2)
|
||||||
@ -89,88 +95,52 @@ const ResponseTags = ({
|
|||||||
<ChatBoxDivider icon="core/chat/quoteFill" text={t('chat.Quote')} />
|
<ChatBoxDivider icon="core/chat/quoteFill" text={t('chat.Quote')} />
|
||||||
<Flex alignItems={'center'} flexWrap={'wrap'} gap={2}>
|
<Flex alignItems={'center'} flexWrap={'wrap'} gap={2}>
|
||||||
{sourceList.map((item) => (
|
{sourceList.map((item) => (
|
||||||
<Flex
|
<MyTooltip key={item.sourceName} label={t('core.chat.quote.Read Quote')}>
|
||||||
key={item.sourceName}
|
<Flex
|
||||||
alignItems={'center'}
|
|
||||||
flexWrap={'wrap'}
|
|
||||||
fontSize={'sm'}
|
|
||||||
border={theme.borders.sm}
|
|
||||||
py={1}
|
|
||||||
px={2}
|
|
||||||
borderRadius={'md'}
|
|
||||||
_hover={{
|
|
||||||
'.controller': {
|
|
||||||
display: 'flex'
|
|
||||||
}
|
|
||||||
}}
|
|
||||||
overflow={'hidden'}
|
|
||||||
position={'relative'}
|
|
||||||
>
|
|
||||||
<Image src={item.icon} alt={''} mr={1} w={'12px'} />
|
|
||||||
<Box className="textEllipsis" flex={'1 0 0'}>
|
|
||||||
{item.sourceName}
|
|
||||||
</Box>
|
|
||||||
|
|
||||||
<Box
|
|
||||||
className="controller"
|
|
||||||
display={'none'}
|
|
||||||
pr={2}
|
|
||||||
position={'absolute'}
|
|
||||||
right={0}
|
|
||||||
left={0}
|
|
||||||
justifyContent={'flex-end'}
|
|
||||||
alignItems={'center'}
|
alignItems={'center'}
|
||||||
h={'100%'}
|
fontSize={'sm'}
|
||||||
lineHeight={0}
|
border={theme.borders.sm}
|
||||||
bg={`linear-gradient(to left, white,white ${
|
py={1}
|
||||||
item.sourceId ? '60px' : '30px'
|
px={2}
|
||||||
}, rgba(255,255,255,0) 80%)`}
|
borderRadius={'sm'}
|
||||||
|
_hover={{
|
||||||
|
'.controller': {
|
||||||
|
display: 'flex'
|
||||||
|
}
|
||||||
|
}}
|
||||||
|
overflow={'hidden'}
|
||||||
|
position={'relative'}
|
||||||
|
cursor={'pointer'}
|
||||||
|
onClick={(e) => {
|
||||||
|
e.stopPropagation();
|
||||||
|
setQuoteModalData({
|
||||||
|
rawSearch: quoteList,
|
||||||
|
metadata: {
|
||||||
|
collectionId: item.collectionId,
|
||||||
|
sourceId: item.sourceId,
|
||||||
|
sourceName: item.sourceName
|
||||||
|
}
|
||||||
|
});
|
||||||
|
}}
|
||||||
>
|
>
|
||||||
<MyTooltip label={t('core.chat.quote.Read Quote')}>
|
<Image src={item.icon} alt={''} mr={1} flexShrink={0} w={'12px'} />
|
||||||
<MyIcon
|
<Box className="textEllipsis3" wordBreak={'break-all'} flex={'1 0 0'}>
|
||||||
name="common/viewLight"
|
{item.sourceName}
|
||||||
w={'14px'}
|
</Box>
|
||||||
cursor={'pointer'}
|
</Flex>
|
||||||
_hover={{
|
</MyTooltip>
|
||||||
color: 'green.600'
|
|
||||||
}}
|
|
||||||
onClick={(e) => {
|
|
||||||
e.stopPropagation();
|
|
||||||
setQuoteModalData(quoteList);
|
|
||||||
}}
|
|
||||||
/>
|
|
||||||
</MyTooltip>
|
|
||||||
{item.sourceId && item.canReadQuote && (
|
|
||||||
<MyTooltip label={t('core.chat.quote.Read Source')}>
|
|
||||||
<MyIcon
|
|
||||||
ml={4}
|
|
||||||
name="common/routePushLight"
|
|
||||||
w={'14px'}
|
|
||||||
cursor={'pointer'}
|
|
||||||
_hover={{ color: 'primary.500' }}
|
|
||||||
onClick={async (e) => {
|
|
||||||
e.stopPropagation();
|
|
||||||
|
|
||||||
if (!item.sourceId) return;
|
|
||||||
await getFileAndOpen(item.sourceId);
|
|
||||||
}}
|
|
||||||
/>
|
|
||||||
</MyTooltip>
|
|
||||||
)}
|
|
||||||
</Box>
|
|
||||||
</Flex>
|
|
||||||
))}
|
))}
|
||||||
</Flex>
|
</Flex>
|
||||||
</>
|
</>
|
||||||
)}
|
)}
|
||||||
<Flex alignItems={'center'} mt={2} flexWrap={'wrap'}>
|
<Flex alignItems={'center'} mt={3} flexWrap={'wrap'}>
|
||||||
{quoteList.length > 0 && (
|
{quoteList.length > 0 && (
|
||||||
<MyTooltip label="查看引用">
|
<MyTooltip label="查看引用">
|
||||||
<Tag
|
<Tag
|
||||||
colorSchema="blue"
|
colorSchema="blue"
|
||||||
cursor={'pointer'}
|
cursor={'pointer'}
|
||||||
{...TagStyles}
|
{...TagStyles}
|
||||||
onClick={() => setQuoteModalData(quoteList)}
|
onClick={() => setQuoteModalData({ rawSearch: quoteList })}
|
||||||
>
|
>
|
||||||
{quoteList.length}条引用
|
{quoteList.length}条引用
|
||||||
</Tag>
|
</Tag>
|
||||||
@ -213,7 +183,7 @@ const ResponseTags = ({
|
|||||||
|
|
||||||
{!!quoteModalData && (
|
{!!quoteModalData && (
|
||||||
<QuoteModal
|
<QuoteModal
|
||||||
rawSearch={quoteModalData}
|
{...quoteModalData}
|
||||||
isShare={isShare}
|
isShare={isShare}
|
||||||
onClose={() => setQuoteModalData(undefined)}
|
onClose={() => setQuoteModalData(undefined)}
|
||||||
/>
|
/>
|
||||||
|
|||||||
@ -632,14 +632,13 @@ const ChatBox = (
|
|||||||
leftIcon={<MyIcon name={'chatFill'} w={'16px'} />}
|
leftIcon={<MyIcon name={'chatFill'} w={'16px'} />}
|
||||||
size={'sm'}
|
size={'sm'}
|
||||||
maxW={'100px'}
|
maxW={'100px'}
|
||||||
borderRadius={'lg'}
|
|
||||||
onClick={handleSubmit((data) => {
|
onClick={handleSubmit((data) => {
|
||||||
onUpdateVariable?.(data);
|
onUpdateVariable?.(data);
|
||||||
setVariables(data);
|
setVariables(data);
|
||||||
setVariableInputFinish(true);
|
setVariableInputFinish(true);
|
||||||
})}
|
})}
|
||||||
>
|
>
|
||||||
{'开始对话'}
|
{t('core.chat.Start Chat')}
|
||||||
</Button>
|
</Button>
|
||||||
)}
|
)}
|
||||||
</Card>
|
</Card>
|
||||||
@ -952,6 +951,8 @@ const ChatBox = (
|
|||||||
appId={appId}
|
appId={appId}
|
||||||
chatId={chatId}
|
chatId={chatId}
|
||||||
chatItemId={feedbackId}
|
chatItemId={feedbackId}
|
||||||
|
shareId={shareId}
|
||||||
|
outLinkUid={outLinkUid}
|
||||||
onClose={() => setFeedbackId(undefined)}
|
onClose={() => setFeedbackId(undefined)}
|
||||||
onSuccess={(content: string) => {
|
onSuccess={(content: string) => {
|
||||||
setChatHistory((state) =>
|
setChatHistory((state) =>
|
||||||
@ -1142,7 +1143,7 @@ function ChatAvatar({ src, type }: { src?: string; type: 'Human' | 'AI' }) {
|
|||||||
w={['28px', '34px']}
|
w={['28px', '34px']}
|
||||||
h={['28px', '34px']}
|
h={['28px', '34px']}
|
||||||
p={'2px'}
|
p={'2px'}
|
||||||
borderRadius={'lg'}
|
borderRadius={'sm'}
|
||||||
border={theme.borders.base}
|
border={theme.borders.base}
|
||||||
boxShadow={'0 0 5px rgba(0,0,0,0.1)'}
|
boxShadow={'0 0 5px rgba(0,0,0,0.1)'}
|
||||||
bg={type === 'Human' ? 'white' : 'primary.50'}
|
bg={type === 'Human' ? 'white' : 'primary.50'}
|
||||||
@ -1208,7 +1209,7 @@ function ChatController({
|
|||||||
cursor: 'pointer',
|
cursor: 'pointer',
|
||||||
p: 1,
|
p: 1,
|
||||||
bg: 'white',
|
bg: 'white',
|
||||||
borderRadius: 'lg',
|
borderRadius: 'md',
|
||||||
boxShadow: '0 0 5px rgba(0,0,0,0.1)',
|
boxShadow: '0 0 5px rgba(0,0,0,0.1)',
|
||||||
border: theme.borders.base,
|
border: theme.borders.base,
|
||||||
mr: 3
|
mr: 3
|
||||||
|
|||||||
@ -9,7 +9,6 @@ import {
|
|||||||
Skeleton,
|
Skeleton,
|
||||||
useDisclosure
|
useDisclosure
|
||||||
} from '@chakra-ui/react';
|
} from '@chakra-ui/react';
|
||||||
import MyModal from '@/components/MyModal';
|
|
||||||
|
|
||||||
const MdImage = ({ src }: { src?: string }) => {
|
const MdImage = ({ src }: { src?: string }) => {
|
||||||
const [isLoading, setIsLoading] = useState(true);
|
const [isLoading, setIsLoading] = useState(true);
|
||||||
@ -34,6 +33,7 @@ const MdImage = ({ src }: { src?: string }) => {
|
|||||||
cursor={succeed ? 'pointer' : 'default'}
|
cursor={succeed ? 'pointer' : 'default'}
|
||||||
loading="eager"
|
loading="eager"
|
||||||
objectFit={'contain'}
|
objectFit={'contain'}
|
||||||
|
referrerPolicy="no-referrer"
|
||||||
onLoad={() => {
|
onLoad={() => {
|
||||||
setIsLoading(false);
|
setIsLoading(false);
|
||||||
setSucceed(true);
|
setSucceed(true);
|
||||||
@ -53,6 +53,7 @@ const MdImage = ({ src }: { src?: string }) => {
|
|||||||
alt={''}
|
alt={''}
|
||||||
w={'100%'}
|
w={'100%'}
|
||||||
maxH={'80vh'}
|
maxH={'80vh'}
|
||||||
|
referrerPolicy="no-referrer"
|
||||||
fallbackSrc={'/imgs/errImg.png'}
|
fallbackSrc={'/imgs/errImg.png'}
|
||||||
fallbackStrategy={'onError'}
|
fallbackStrategy={'onError'}
|
||||||
objectFit={'contain'}
|
objectFit={'contain'}
|
||||||
|
|||||||
@ -39,7 +39,7 @@ const Tag = ({ children, colorSchema = 'blue', ...props }: Props) => {
|
|||||||
px={2}
|
px={2}
|
||||||
lineHeight={1}
|
lineHeight={1}
|
||||||
py={1}
|
py={1}
|
||||||
borderRadius={'md'}
|
borderRadius={'sm'}
|
||||||
fontSize={'xs'}
|
fontSize={'xs'}
|
||||||
alignItems={'center'}
|
alignItems={'center'}
|
||||||
{...theme}
|
{...theme}
|
||||||
|
|||||||
@ -77,7 +77,7 @@ const Editor = React.memo(function Editor({
|
|||||||
|
|
||||||
return (
|
return (
|
||||||
<Box h={'100%'} w={'100%'} position={'relative'}>
|
<Box h={'100%'} w={'100%'} position={'relative'}>
|
||||||
<Textarea ref={textareaRef} textAlign={'justify'} maxW={'100%'} {...props} />
|
<Textarea ref={textareaRef} maxW={'100%'} {...props} />
|
||||||
{onOpenModal && (
|
{onOpenModal && (
|
||||||
<Box
|
<Box
|
||||||
zIndex={1}
|
zIndex={1}
|
||||||
|
|||||||
223
projects/app/src/components/core/dataset/QuoteItem.tsx
Normal file
223
projects/app/src/components/core/dataset/QuoteItem.tsx
Normal file
@ -0,0 +1,223 @@
|
|||||||
|
import React, { useMemo, useState } from 'react';
|
||||||
|
import { Box, Flex, Link, Progress, useTheme } from '@chakra-ui/react';
|
||||||
|
import {
|
||||||
|
type InputDataType,
|
||||||
|
RawSourceText
|
||||||
|
} from '@/pages/dataset/detail/components/InputDataModal';
|
||||||
|
import type { SearchDataResponseItemType } from '@fastgpt/global/core/dataset/type.d';
|
||||||
|
import NextLink from 'next/link';
|
||||||
|
import MyIcon from '@/components/Icon';
|
||||||
|
import { useTranslation } from 'next-i18next';
|
||||||
|
import MyTooltip from '@/components/MyTooltip';
|
||||||
|
import { useSystemStore } from '@/web/common/system/useSystemStore';
|
||||||
|
import dynamic from 'next/dynamic';
|
||||||
|
import MyBox from '@/components/common/MyBox';
|
||||||
|
import { getDatasetDataItemById } from '@/web/core/dataset/api';
|
||||||
|
import { useRequest } from '@/web/common/hooks/useRequest';
|
||||||
|
import { DatasetDataItemType } from '@fastgpt/global/core/dataset/type';
|
||||||
|
import { SearchScoreTypeEnum, SearchScoreTypeMap } from '@fastgpt/global/core/dataset/constant';
|
||||||
|
|
||||||
|
const InputDataModal = dynamic(() => import('@/pages/dataset/detail/components/InputDataModal'));
|
||||||
|
|
||||||
|
const QuoteItem = ({
|
||||||
|
quoteItem,
|
||||||
|
canViewSource,
|
||||||
|
linkToDataset
|
||||||
|
}: {
|
||||||
|
quoteItem: SearchDataResponseItemType;
|
||||||
|
canViewSource?: boolean;
|
||||||
|
linkToDataset?: boolean;
|
||||||
|
}) => {
|
||||||
|
const { t } = useTranslation();
|
||||||
|
const { isPc } = useSystemStore();
|
||||||
|
const theme = useTheme();
|
||||||
|
const [editInputData, setEditInputData] = useState<InputDataType & { collectionId: string }>();
|
||||||
|
|
||||||
|
const { mutate: onclickEdit, isLoading } = useRequest({
|
||||||
|
mutationFn: async (id: string) => {
|
||||||
|
return getDatasetDataItemById(id);
|
||||||
|
},
|
||||||
|
onSuccess(data: DatasetDataItemType) {
|
||||||
|
setEditInputData(data);
|
||||||
|
},
|
||||||
|
errorToast: t('core.dataset.data.get data error')
|
||||||
|
});
|
||||||
|
|
||||||
|
const rank = useMemo(() => {
|
||||||
|
if (quoteItem.score.length === 1) {
|
||||||
|
return quoteItem.score[0].index;
|
||||||
|
}
|
||||||
|
const rrf = quoteItem.score?.find((item) => item.type === SearchScoreTypeEnum.rrf);
|
||||||
|
if (rrf) return rrf.index;
|
||||||
|
|
||||||
|
return 0;
|
||||||
|
}, [quoteItem.score]);
|
||||||
|
|
||||||
|
const score = useMemo(() => {
|
||||||
|
let searchScore: number | undefined = undefined;
|
||||||
|
let text = '';
|
||||||
|
|
||||||
|
const reRankScore = quoteItem.score?.find((item) => item.type === SearchScoreTypeEnum.reRank);
|
||||||
|
if (reRankScore) {
|
||||||
|
searchScore = reRankScore.value;
|
||||||
|
text = t('core.dataset.search.Rerank score');
|
||||||
|
}
|
||||||
|
|
||||||
|
const embScore = quoteItem.score?.find((item) => item.type === SearchScoreTypeEnum.embedding);
|
||||||
|
if (embScore && quoteItem.score.length === 1) {
|
||||||
|
searchScore = embScore.value;
|
||||||
|
text = t('core.dataset.search.Embedding score');
|
||||||
|
}
|
||||||
|
|
||||||
|
const detailScore = (() => {
|
||||||
|
if (Array.isArray(quoteItem.score)) {
|
||||||
|
return quoteItem.score
|
||||||
|
.map(
|
||||||
|
(item) =>
|
||||||
|
`${t('core.dataset.search.Search type')}: ${t(SearchScoreTypeMap[item.type]?.label)}
|
||||||
|
${t('core.dataset.search.Rank')}: ${item.index + 1}
|
||||||
|
${t('core.dataset.search.Score')}: ${item.value.toFixed(4)}`
|
||||||
|
)
|
||||||
|
.join('\n----\n');
|
||||||
|
}
|
||||||
|
return 'null';
|
||||||
|
})();
|
||||||
|
|
||||||
|
return {
|
||||||
|
value: searchScore,
|
||||||
|
tip: t('core.dataset.Search score tip', {
|
||||||
|
scoreText: text ? `${text}。\n` : text,
|
||||||
|
detailScore
|
||||||
|
})
|
||||||
|
};
|
||||||
|
}, [quoteItem.score, t]);
|
||||||
|
|
||||||
|
return (
|
||||||
|
<>
|
||||||
|
<MyBox
|
||||||
|
isLoading={isLoading}
|
||||||
|
position={'relative'}
|
||||||
|
overflow={'hidden'}
|
||||||
|
fontSize={'sm'}
|
||||||
|
whiteSpace={'pre-wrap'}
|
||||||
|
_hover={{ '& .hover-data': { display: 'flex' } }}
|
||||||
|
>
|
||||||
|
<Flex alignItems={'flex-end'} mb={3}>
|
||||||
|
{rank !== undefined && (
|
||||||
|
<MyTooltip label={t('core.dataset.search.Rank Tip')}>
|
||||||
|
<Box px={2} py={'3px'} mr={3} bg={'myGray.200'} borderRadius={'md'}>
|
||||||
|
# {rank + 1}
|
||||||
|
</Box>
|
||||||
|
</MyTooltip>
|
||||||
|
)}
|
||||||
|
|
||||||
|
<RawSourceText
|
||||||
|
fontWeight={'bold'}
|
||||||
|
color={'black'}
|
||||||
|
sourceName={quoteItem.sourceName}
|
||||||
|
sourceId={quoteItem.sourceId}
|
||||||
|
canView={canViewSource}
|
||||||
|
/>
|
||||||
|
<Box flex={1} />
|
||||||
|
{linkToDataset && (
|
||||||
|
<Link
|
||||||
|
as={NextLink}
|
||||||
|
className="hover-data"
|
||||||
|
display={'none'}
|
||||||
|
alignItems={'center'}
|
||||||
|
color={'primary.500'}
|
||||||
|
href={`/dataset/detail?datasetId=${quoteItem.datasetId}¤tTab=dataCard&collectionId=${quoteItem.collectionId}`}
|
||||||
|
>
|
||||||
|
{t('core.dataset.Go Dataset')}
|
||||||
|
<MyIcon name={'common/rightArrowLight'} w={'10px'} />
|
||||||
|
</Link>
|
||||||
|
)}
|
||||||
|
</Flex>
|
||||||
|
|
||||||
|
<Box color={'black'}>{quoteItem.q}</Box>
|
||||||
|
<Box color={'myGray.600'}>{quoteItem.a}</Box>
|
||||||
|
{canViewSource && (
|
||||||
|
<Flex alignItems={'center'} mt={3} gap={4} color={'myGray.500'} fontSize={'xs'}>
|
||||||
|
{isPc && (
|
||||||
|
<Flex border={theme.borders.base} px={3} borderRadius={'xs'} lineHeight={'16px'}>
|
||||||
|
ID: {quoteItem.id}
|
||||||
|
</Flex>
|
||||||
|
)}
|
||||||
|
<MyTooltip label={t('core.dataset.Quote Length')}>
|
||||||
|
<Flex alignItems={'center'}>
|
||||||
|
<MyIcon name="common/text/t" w={'14px'} mr={1} color={'myGray.500'} />
|
||||||
|
{quoteItem.q.length + (quoteItem.a?.length || 0)}
|
||||||
|
</Flex>
|
||||||
|
</MyTooltip>
|
||||||
|
{canViewSource && score && (
|
||||||
|
<MyTooltip label={score.tip}>
|
||||||
|
<Flex alignItems={'center'}>
|
||||||
|
<MyIcon name={'kbTest'} w={'12px'} />
|
||||||
|
{score.value ? (
|
||||||
|
<>
|
||||||
|
<Progress
|
||||||
|
mx={2}
|
||||||
|
w={['60px', '90px']}
|
||||||
|
value={score?.value * 100}
|
||||||
|
size="sm"
|
||||||
|
borderRadius={'20px'}
|
||||||
|
colorScheme="myGray"
|
||||||
|
border={theme.borders.base}
|
||||||
|
/>
|
||||||
|
<Box>{score?.value.toFixed(4)}</Box>
|
||||||
|
</>
|
||||||
|
) : (
|
||||||
|
<Box ml={1} cursor={'pointer'}>
|
||||||
|
{t('core.dataset.search.Read score')}
|
||||||
|
</Box>
|
||||||
|
)}
|
||||||
|
</Flex>
|
||||||
|
</MyTooltip>
|
||||||
|
)}
|
||||||
|
<Box flex={1} />
|
||||||
|
{quoteItem.id && (
|
||||||
|
<MyTooltip label={t('core.dataset.data.Edit')}>
|
||||||
|
<Box
|
||||||
|
className="hover-data"
|
||||||
|
display={['flex', 'none']}
|
||||||
|
bg={'rgba(255,255,255,0.9)'}
|
||||||
|
alignItems={'center'}
|
||||||
|
justifyContent={'center'}
|
||||||
|
boxShadow={'-10px 0 10px rgba(255,255,255,1)'}
|
||||||
|
>
|
||||||
|
<MyIcon
|
||||||
|
name={'edit'}
|
||||||
|
w={['16px', '18px']}
|
||||||
|
h={['16px', '18px']}
|
||||||
|
cursor={'pointer'}
|
||||||
|
color={'myGray.600'}
|
||||||
|
_hover={{
|
||||||
|
color: 'primary.600'
|
||||||
|
}}
|
||||||
|
onClick={() => onclickEdit(quoteItem.id)}
|
||||||
|
/>
|
||||||
|
</Box>
|
||||||
|
</MyTooltip>
|
||||||
|
)}
|
||||||
|
</Flex>
|
||||||
|
)}
|
||||||
|
</MyBox>
|
||||||
|
|
||||||
|
{editInputData && editInputData.id && (
|
||||||
|
<InputDataModal
|
||||||
|
onClose={() => setEditInputData(undefined)}
|
||||||
|
onSuccess={() => {
|
||||||
|
console.log('更新引用成功');
|
||||||
|
}}
|
||||||
|
onDelete={() => {
|
||||||
|
console.log('删除引用成功');
|
||||||
|
}}
|
||||||
|
defaultValue={editInputData}
|
||||||
|
collectionId={editInputData.collectionId}
|
||||||
|
/>
|
||||||
|
)}
|
||||||
|
</>
|
||||||
|
);
|
||||||
|
};
|
||||||
|
|
||||||
|
export default React.memo(QuoteItem);
|
||||||
@ -128,7 +128,7 @@ const VariableEdit = ({
|
|||||||
</Flex>
|
</Flex>
|
||||||
</Flex>
|
</Flex>
|
||||||
{formatVariables.length > 0 && (
|
{formatVariables.length > 0 && (
|
||||||
<Box mt={2} borderRadius={'lg'} overflow={'hidden'} borderWidth={'1px'} borderBottom="none">
|
<Box mt={2} borderRadius={'md'} overflow={'hidden'} borderWidth={'1px'} borderBottom="none">
|
||||||
<TableContainer>
|
<TableContainer>
|
||||||
<Table bg={'white'}>
|
<Table bg={'white'}>
|
||||||
<Thead>
|
<Thead>
|
||||||
@ -266,7 +266,7 @@ const VariableEdit = ({
|
|||||||
w={'16px'}
|
w={'16px'}
|
||||||
cursor={'pointer'}
|
cursor={'pointer'}
|
||||||
p={2}
|
p={2}
|
||||||
borderRadius={'lg'}
|
borderRadius={'md'}
|
||||||
_hover={{ bg: 'red.100' }}
|
_hover={{ bg: 'red.100' }}
|
||||||
onClick={() => removeEnums(i)}
|
onClick={() => removeEnums(i)}
|
||||||
/>
|
/>
|
||||||
|
|||||||
@ -7,6 +7,7 @@ import dynamic from 'next/dynamic';
|
|||||||
import InputLabel from './Label';
|
import InputLabel from './Label';
|
||||||
import type { RenderInputProps } from './type.d';
|
import type { RenderInputProps } from './type.d';
|
||||||
import { getFlowStore, type useFlowProviderStoreType } from '../../../FlowProvider';
|
import { getFlowStore, type useFlowProviderStoreType } from '../../../FlowProvider';
|
||||||
|
import { ModuleInputKeyEnum } from '@fastgpt/global/core/module/constants';
|
||||||
|
|
||||||
const RenderList: {
|
const RenderList: {
|
||||||
types: `${FlowNodeInputTypeEnum}`[];
|
types: `${FlowNodeInputTypeEnum}`[];
|
||||||
@ -65,6 +66,7 @@ const RenderList: {
|
|||||||
Component: dynamic(() => import('./templates/AddInputParam'))
|
Component: dynamic(() => import('./templates/AddInputParam'))
|
||||||
}
|
}
|
||||||
];
|
];
|
||||||
|
const UserChatInput = dynamic(() => import('./templates/UserChatInput'));
|
||||||
|
|
||||||
type Props = {
|
type Props = {
|
||||||
flowInputList: FlowNodeInputItemType[];
|
flowInputList: FlowNodeInputItemType[];
|
||||||
@ -124,18 +126,23 @@ const RenderInput = ({ flowInputList, moduleId, CustomComponent = {} }: Props) =
|
|||||||
})();
|
})();
|
||||||
|
|
||||||
return (
|
return (
|
||||||
input.type !== FlowNodeInputTypeEnum.hidden && (
|
<Box key={input.key} _notLast={{ mb: 7 }} position={'relative'}>
|
||||||
<Box key={input.key} _notLast={{ mb: 7 }} position={'relative'}>
|
{input.key === ModuleInputKeyEnum.userChatInput && (
|
||||||
{!!input.label && (
|
<UserChatInput inputs={filterInputs} item={input} moduleId={moduleId} />
|
||||||
<InputLabel moduleId={moduleId} inputKey={input.key} mode={mode} {...input} />
|
)}
|
||||||
)}
|
{input.type !== FlowNodeInputTypeEnum.hidden && (
|
||||||
{!!RenderComponent && (
|
<>
|
||||||
<Box mt={2} className={'nodrag'}>
|
{!!input.label && (
|
||||||
{RenderComponent}
|
<InputLabel moduleId={moduleId} inputKey={input.key} mode={mode} {...input} />
|
||||||
</Box>
|
)}
|
||||||
)}
|
{!!RenderComponent && (
|
||||||
</Box>
|
<Box mt={2} className={'nodrag'}>
|
||||||
)
|
{RenderComponent}
|
||||||
|
</Box>
|
||||||
|
)}
|
||||||
|
</>
|
||||||
|
)}
|
||||||
|
</Box>
|
||||||
);
|
);
|
||||||
})}
|
})}
|
||||||
</>
|
</>
|
||||||
|
|||||||
@ -0,0 +1,41 @@
|
|||||||
|
import React from 'react';
|
||||||
|
import type { RenderInputProps } from '../type';
|
||||||
|
import { Box, Flex } from '@chakra-ui/react';
|
||||||
|
import { useTranslation } from 'next-i18next';
|
||||||
|
import TargetHandle from '../../TargetHandle';
|
||||||
|
import SourceHandle from '../../SourceHandle';
|
||||||
|
import { ModuleInputKeyEnum, ModuleOutputKeyEnum } from '@fastgpt/global/core/module/constants';
|
||||||
|
|
||||||
|
const UserChatInput = ({ item }: RenderInputProps) => {
|
||||||
|
const { t } = useTranslation();
|
||||||
|
|
||||||
|
return (
|
||||||
|
<Flex
|
||||||
|
className="nodrag"
|
||||||
|
cursor={'default'}
|
||||||
|
alignItems={'center'}
|
||||||
|
justifyContent={'space-between'}
|
||||||
|
position={'relative'}
|
||||||
|
>
|
||||||
|
<Box position={'relative'}>
|
||||||
|
<TargetHandle handleKey={ModuleInputKeyEnum.userChatInput} valueType={item.valueType} />
|
||||||
|
{t('core.module.input.label.user question')}
|
||||||
|
<Box
|
||||||
|
position={'absolute'}
|
||||||
|
top={'-2px'}
|
||||||
|
right={'-8px'}
|
||||||
|
color={'red.500'}
|
||||||
|
fontWeight={'bold'}
|
||||||
|
>
|
||||||
|
*
|
||||||
|
</Box>
|
||||||
|
</Box>
|
||||||
|
<Box position={'relative'}>
|
||||||
|
{t('core.module.input.label.user question')}
|
||||||
|
<SourceHandle handleKey={ModuleOutputKeyEnum.userChatInput} valueType={item.valueType} />
|
||||||
|
</Box>
|
||||||
|
</Flex>
|
||||||
|
);
|
||||||
|
};
|
||||||
|
|
||||||
|
export default React.memo(UserChatInput);
|
||||||
@ -59,8 +59,13 @@ export type SearchTestProps = {
|
|||||||
limit?: number;
|
limit?: number;
|
||||||
searchMode?: `${DatasetSearchModeEnum}`;
|
searchMode?: `${DatasetSearchModeEnum}`;
|
||||||
usingReRank: boolean;
|
usingReRank: boolean;
|
||||||
|
similarity?: number;
|
||||||
};
|
};
|
||||||
export type SearchTestResponse = {
|
export type SearchTestResponse = {
|
||||||
list: SearchDataResponseItemType[];
|
list: SearchDataResponseItemType[];
|
||||||
duration: string;
|
duration: string;
|
||||||
|
limit: number;
|
||||||
|
searchMode: `${DatasetSearchModeEnum}`;
|
||||||
|
usingReRank: boolean;
|
||||||
|
similarity: number;
|
||||||
};
|
};
|
||||||
|
|||||||
@ -36,11 +36,7 @@ export const Prompt_ExtractJson = `你可以从 <对话记录></对话记录>
|
|||||||
</对话记录>
|
</对话记录>
|
||||||
`;
|
`;
|
||||||
|
|
||||||
export const Prompt_CQJson = `我会给你几个问题类型,请参考额外的背景知识(可能为空)和对话内容,判断我本次的问题类型,并返回对应类型的 ID,格式为 JSON 字符串:
|
export const Prompt_CQJson = `我会给你几个问题类型,请参考背景知识(可能为空)和对话记录,判断我“本次问题”的类型,并返回一个问题“类型ID”:
|
||||||
"""
|
|
||||||
'{"问题类型":"类型的 ID"}'
|
|
||||||
"""
|
|
||||||
|
|
||||||
<问题类型>
|
<问题类型>
|
||||||
{{typeList}}
|
{{typeList}}
|
||||||
</问题类型>
|
</问题类型>
|
||||||
@ -49,9 +45,13 @@ export const Prompt_CQJson = `我会给你几个问题类型,请参考额外
|
|||||||
{{systemPrompt}}
|
{{systemPrompt}}
|
||||||
</背景知识>
|
</背景知识>
|
||||||
|
|
||||||
<对话内容>
|
<对话记录>
|
||||||
{{text}}
|
{{history}}
|
||||||
</对话内容>
|
</对话记录>
|
||||||
|
|
||||||
|
Human:"{{question}}"
|
||||||
|
|
||||||
|
类型ID=
|
||||||
`;
|
`;
|
||||||
|
|
||||||
export const Prompt_QuestionGuide = `我不太清楚问你什么问题,请帮我生成 3 个问题,引导我继续提问。问题的长度应小于20个字符,按 JSON 格式返回: ["问题1", "问题2", "问题3"]`;
|
export const Prompt_QuestionGuide = `我不太清楚问你什么问题,请帮我生成 3 个问题,引导我继续提问。问题的长度应小于20个字符,按 JSON 格式返回: ["问题1", "问题2", "问题3"]`;
|
||||||
|
|||||||
@ -14,7 +14,14 @@ import { searchQueryExtension } from '@fastgpt/service/core/ai/functions/queryEx
|
|||||||
export default withNextCors(async function handler(req: NextApiRequest, res: NextApiResponse<any>) {
|
export default withNextCors(async function handler(req: NextApiRequest, res: NextApiResponse<any>) {
|
||||||
try {
|
try {
|
||||||
await connectToDatabase();
|
await connectToDatabase();
|
||||||
const { datasetId, text, limit = 20, searchMode, usingReRank } = req.body as SearchTestProps;
|
const {
|
||||||
|
datasetId,
|
||||||
|
text,
|
||||||
|
limit = 1500,
|
||||||
|
similarity,
|
||||||
|
searchMode,
|
||||||
|
usingReRank
|
||||||
|
} = req.body as SearchTestProps;
|
||||||
|
|
||||||
if (!datasetId || !text) {
|
if (!datasetId || !text) {
|
||||||
throw new Error('缺少参数');
|
throw new Error('缺少参数');
|
||||||
@ -40,11 +47,12 @@ export default withNextCors(async function handler(req: NextApiRequest, res: Nex
|
|||||||
// model: global.chatModels[0].model
|
// model: global.chatModels[0].model
|
||||||
// });
|
// });
|
||||||
|
|
||||||
const { searchRes, tokens } = await searchDatasetData({
|
const { searchRes, tokens, ...result } = await searchDatasetData({
|
||||||
rawQuery: text,
|
rawQuery: text,
|
||||||
queries: [text],
|
queries: [text],
|
||||||
model: dataset.vectorModel,
|
model: dataset.vectorModel,
|
||||||
limit: Math.min(limit * 800, 30000),
|
limit: Math.min(limit, 20000),
|
||||||
|
similarity,
|
||||||
datasetIds: [datasetId],
|
datasetIds: [datasetId],
|
||||||
searchMode,
|
searchMode,
|
||||||
usingReRank
|
usingReRank
|
||||||
@ -68,7 +76,8 @@ export default withNextCors(async function handler(req: NextApiRequest, res: Nex
|
|||||||
jsonRes<SearchTestResponse>(res, {
|
jsonRes<SearchTestResponse>(res, {
|
||||||
data: {
|
data: {
|
||||||
list: searchRes,
|
list: searchRes,
|
||||||
duration: `${((Date.now() - start) / 1000).toFixed(3)}s`
|
duration: `${((Date.now() - start) / 1000).toFixed(3)}s`,
|
||||||
|
...result
|
||||||
}
|
}
|
||||||
});
|
});
|
||||||
} catch (err) {
|
} catch (err) {
|
||||||
|
|||||||
@ -133,7 +133,7 @@ const InfoModal = ({
|
|||||||
w={['26px', '34px']}
|
w={['26px', '34px']}
|
||||||
h={['26px', '34px']}
|
h={['26px', '34px']}
|
||||||
cursor={'pointer'}
|
cursor={'pointer'}
|
||||||
borderRadius={'lg'}
|
borderRadius={'md'}
|
||||||
mr={4}
|
mr={4}
|
||||||
title={'点击切换头像'}
|
title={'点击切换头像'}
|
||||||
onClick={() => onOpenSelectFile()}
|
onClick={() => onOpenSelectFile()}
|
||||||
|
|||||||
@ -128,7 +128,7 @@ const Logs = ({ appId }: { appId: string }) => {
|
|||||||
py={1}
|
py={1}
|
||||||
alignItems={'center'}
|
alignItems={'center'}
|
||||||
justifyContent={'center'}
|
justifyContent={'center'}
|
||||||
borderRadius={'lg'}
|
borderRadius={'md'}
|
||||||
fontWeight={'bold'}
|
fontWeight={'bold'}
|
||||||
>
|
>
|
||||||
<MyIcon
|
<MyIcon
|
||||||
@ -148,7 +148,7 @@ const Logs = ({ appId }: { appId: string }) => {
|
|||||||
py={1}
|
py={1}
|
||||||
alignItems={'center'}
|
alignItems={'center'}
|
||||||
justifyContent={'center'}
|
justifyContent={'center'}
|
||||||
borderRadius={'lg'}
|
borderRadius={'md'}
|
||||||
fontWeight={'bold'}
|
fontWeight={'bold'}
|
||||||
>
|
>
|
||||||
<MyIcon
|
<MyIcon
|
||||||
|
|||||||
@ -20,7 +20,7 @@ const OutLink = ({ appId }: { appId: string }) => {
|
|||||||
</Box>
|
</Box>
|
||||||
<Box pb={[5, 7]} px={[4, 8]} borderBottom={theme.borders.base}>
|
<Box pb={[5, 7]} px={[4, 8]} borderBottom={theme.borders.base}>
|
||||||
<MyRadio
|
<MyRadio
|
||||||
gridTemplateColumns={['repeat(1,1fr)', 'repeat(auto-fill, minmax(0, 360px))']}
|
gridTemplateColumns={['repeat(1,1fr)', 'repeat(auto-fill, minmax(0, 400px))']}
|
||||||
iconSize={'20px'}
|
iconSize={'20px'}
|
||||||
list={[
|
list={[
|
||||||
{
|
{
|
||||||
|
|||||||
@ -597,7 +597,7 @@ function Settings({ appId }: { appId: string }) {
|
|||||||
<Box
|
<Box
|
||||||
borderWidth={'1px'}
|
borderWidth={'1px'}
|
||||||
borderColor={'primary.1'}
|
borderColor={'primary.1'}
|
||||||
borderRadius={'lg'}
|
borderRadius={'md'}
|
||||||
mt={2}
|
mt={2}
|
||||||
px={5}
|
px={5}
|
||||||
py={4}
|
py={4}
|
||||||
|
|||||||
@ -108,7 +108,7 @@ const AppDetail = ({ currentTab }: { currentTab: `${TabEnum}` }) => {
|
|||||||
borderRight={theme.borders.base}
|
borderRight={theme.borders.base}
|
||||||
>
|
>
|
||||||
<Flex mb={4} alignItems={'center'}>
|
<Flex mb={4} alignItems={'center'}>
|
||||||
<Avatar src={appDetail.avatar} w={'34px'} borderRadius={'lg'} />
|
<Avatar src={appDetail.avatar} w={'34px'} borderRadius={'md'} />
|
||||||
<Box ml={2} fontWeight={'bold'}>
|
<Box ml={2} fontWeight={'bold'}>
|
||||||
{appDetail.name}
|
{appDetail.name}
|
||||||
</Box>
|
</Box>
|
||||||
|
|||||||
@ -99,7 +99,7 @@ const MyApps = () => {
|
|||||||
borderWidth={'1.5px'}
|
borderWidth={'1.5px'}
|
||||||
borderColor={'borderColor.low'}
|
borderColor={'borderColor.low'}
|
||||||
bg={'white'}
|
bg={'white'}
|
||||||
borderRadius={'lg'}
|
borderRadius={'md'}
|
||||||
userSelect={'none'}
|
userSelect={'none'}
|
||||||
position={'relative'}
|
position={'relative'}
|
||||||
display={'flex'}
|
display={'flex'}
|
||||||
|
|||||||
@ -170,7 +170,7 @@ const ChatHistorySlider = ({
|
|||||||
variant={'whiteDanger'}
|
variant={'whiteDanger'}
|
||||||
size={'mdSquare'}
|
size={'mdSquare'}
|
||||||
aria-label={''}
|
aria-label={''}
|
||||||
borderRadius={'xl'}
|
borderRadius={'50%'}
|
||||||
onClick={openConfirm(onClearHistory)}
|
onClick={openConfirm(onClearHistory)}
|
||||||
>
|
>
|
||||||
<MyIcon name={'clear'} w={'16px'} />
|
<MyIcon name={'clear'} w={'16px'} />
|
||||||
@ -191,7 +191,7 @@ const ChatHistorySlider = ({
|
|||||||
px={4}
|
px={4}
|
||||||
cursor={'pointer'}
|
cursor={'pointer'}
|
||||||
userSelect={'none'}
|
userSelect={'none'}
|
||||||
borderRadius={'lg'}
|
borderRadius={'md'}
|
||||||
mb={2}
|
mb={2}
|
||||||
_hover={{
|
_hover={{
|
||||||
bg: 'myGray.100',
|
bg: 'myGray.100',
|
||||||
@ -287,7 +287,7 @@ const ChatHistorySlider = ({
|
|||||||
py={2}
|
py={2}
|
||||||
px={3}
|
px={3}
|
||||||
mb={3}
|
mb={3}
|
||||||
borderRadius={'lg'}
|
borderRadius={'md'}
|
||||||
alignItems={'center'}
|
alignItems={'center'}
|
||||||
{...(item._id === appId
|
{...(item._id === appId
|
||||||
? {
|
? {
|
||||||
|
|||||||
@ -46,7 +46,7 @@ const SliderApps = ({ appId }: { appId: string }) => {
|
|||||||
px={3}
|
px={3}
|
||||||
mb={3}
|
mb={3}
|
||||||
cursor={'pointer'}
|
cursor={'pointer'}
|
||||||
borderRadius={'lg'}
|
borderRadius={'md'}
|
||||||
alignItems={'center'}
|
alignItems={'center'}
|
||||||
{...(item._id === appId
|
{...(item._id === appId
|
||||||
? {
|
? {
|
||||||
|
|||||||
@ -380,7 +380,7 @@ const FileSelect = ({
|
|||||||
textAlign={'center'}
|
textAlign={'center'}
|
||||||
bg={'myWhite.400'}
|
bg={'myWhite.400'}
|
||||||
p={5}
|
p={5}
|
||||||
borderRadius={'lg'}
|
borderRadius={'md'}
|
||||||
border={'1px dashed'}
|
border={'1px dashed'}
|
||||||
borderColor={'myGray.300'}
|
borderColor={'myGray.300'}
|
||||||
w={'100%'}
|
w={'100%'}
|
||||||
|
|||||||
@ -1,13 +1,26 @@
|
|||||||
import React, { useEffect, useMemo, useState } from 'react';
|
import React, { useEffect, useMemo, useState } from 'react';
|
||||||
import { Box, Textarea, Button, Flex, useTheme, Grid, useDisclosure } from '@chakra-ui/react';
|
import {
|
||||||
|
Box,
|
||||||
|
Textarea,
|
||||||
|
Button,
|
||||||
|
Flex,
|
||||||
|
useTheme,
|
||||||
|
Grid,
|
||||||
|
useDisclosure,
|
||||||
|
Table,
|
||||||
|
Thead,
|
||||||
|
Tbody,
|
||||||
|
Tr,
|
||||||
|
Th,
|
||||||
|
Td,
|
||||||
|
TableContainer
|
||||||
|
} from '@chakra-ui/react';
|
||||||
import { useDatasetStore } from '@/web/core/dataset/store/dataset';
|
import { useDatasetStore } from '@/web/core/dataset/store/dataset';
|
||||||
import { useSearchTestStore, SearchTestStoreItemType } from '@/web/core/dataset/store/searchTest';
|
import { useSearchTestStore, SearchTestStoreItemType } from '@/web/core/dataset/store/searchTest';
|
||||||
import { getDatasetDataItemById, postSearchText } from '@/web/core/dataset/api';
|
import { postSearchText } from '@/web/core/dataset/api';
|
||||||
import MyIcon from '@/components/Icon';
|
import MyIcon from '@/components/Icon';
|
||||||
import { useRequest } from '@/web/common/hooks/useRequest';
|
import { useRequest } from '@/web/common/hooks/useRequest';
|
||||||
import { formatTimeToChatTime } from '@/utils/tools';
|
import { formatTimeToChatTime } from '@/utils/tools';
|
||||||
import InputDataModal, { RawSourceText, type InputDataType } from './InputDataModal';
|
|
||||||
import { useSystemStore } from '@/web/common/system/useSystemStore';
|
|
||||||
import { getErrText } from '@fastgpt/global/common/error/utils';
|
import { getErrText } from '@fastgpt/global/common/error/utils';
|
||||||
import { useToast } from '@/web/common/hooks/useToast';
|
import { useToast } from '@/web/common/hooks/useToast';
|
||||||
import { customAlphabet } from 'nanoid';
|
import { customAlphabet } from 'nanoid';
|
||||||
@ -17,26 +30,55 @@ import { useTranslation } from 'next-i18next';
|
|||||||
import { SearchTestResponse } from '@/global/core/dataset/api';
|
import { SearchTestResponse } from '@/global/core/dataset/api';
|
||||||
import { DatasetSearchModeEnum, DatasetSearchModeMap } from '@fastgpt/global/core/dataset/constant';
|
import { DatasetSearchModeEnum, DatasetSearchModeMap } from '@fastgpt/global/core/dataset/constant';
|
||||||
import dynamic from 'next/dynamic';
|
import dynamic from 'next/dynamic';
|
||||||
|
import { useForm } from 'react-hook-form';
|
||||||
|
import MySelect from '@/components/Select';
|
||||||
|
import { useSelectFile } from '@/web/common/file/hooks/useSelectFile';
|
||||||
|
import { fileDownload, readCsvContent } from '@/web/common/file/utils';
|
||||||
|
import { delay } from '@fastgpt/global/common/system/utils';
|
||||||
|
import QuoteItem from '@/components/core/dataset/QuoteItem';
|
||||||
|
|
||||||
const nanoid = customAlphabet('abcdefghijklmnopqrstuvwxyz1234567890', 12);
|
const nanoid = customAlphabet('abcdefghijklmnopqrstuvwxyz1234567890', 12);
|
||||||
|
|
||||||
const DatasetParamsModal = dynamic(() => import('@/components/core/module/DatasetParamsModal'));
|
const DatasetParamsModal = dynamic(() => import('@/components/core/module/DatasetParamsModal'));
|
||||||
|
|
||||||
|
type FormType = {
|
||||||
|
inputText: string;
|
||||||
|
searchParams: {
|
||||||
|
searchMode: `${DatasetSearchModeEnum}`;
|
||||||
|
usingReRank: boolean;
|
||||||
|
limit: number;
|
||||||
|
similarity: number;
|
||||||
|
};
|
||||||
|
};
|
||||||
|
|
||||||
const Test = ({ datasetId }: { datasetId: string }) => {
|
const Test = ({ datasetId }: { datasetId: string }) => {
|
||||||
const { t } = useTranslation();
|
const { t } = useTranslation();
|
||||||
const theme = useTheme();
|
const theme = useTheme();
|
||||||
const { toast } = useToast();
|
const { toast } = useToast();
|
||||||
const { setLoading } = useSystemStore();
|
|
||||||
const { datasetDetail } = useDatasetStore();
|
const { datasetDetail } = useDatasetStore();
|
||||||
const { datasetTestList, pushDatasetTestItem, delDatasetTestItemById, updateDatasetItemById } =
|
const { pushDatasetTestItem } = useSearchTestStore();
|
||||||
useSearchTestStore();
|
const [inputType, setInputType] = useState<'text' | 'file'>('text');
|
||||||
const [inputText, setInputText] = useState('');
|
|
||||||
const [datasetTestItem, setDatasetTestItem] = useState<SearchTestStoreItemType>();
|
const [datasetTestItem, setDatasetTestItem] = useState<SearchTestStoreItemType>();
|
||||||
const [editInputData, setEditInputData] = useState<InputDataType & { collectionId: string }>();
|
const [refresh, setRefresh] = useState(false);
|
||||||
const [searchMode, setSearchMode] = useState<`${DatasetSearchModeEnum}`>(
|
const { File, onOpen } = useSelectFile({
|
||||||
DatasetSearchModeEnum.embedding
|
fileType: '.csv',
|
||||||
);
|
multiple: false
|
||||||
const [usingReRank, setUsingReRank] = useState(false);
|
});
|
||||||
const searchModeData = DatasetSearchModeMap[searchMode];
|
const [selectFile, setSelectFile] = useState<File>();
|
||||||
|
|
||||||
|
const { getValues, setValue, register, handleSubmit } = useForm<FormType>({
|
||||||
|
defaultValues: {
|
||||||
|
inputText: '',
|
||||||
|
searchParams: {
|
||||||
|
searchMode: DatasetSearchModeEnum.embedding,
|
||||||
|
usingReRank: false,
|
||||||
|
limit: 5000,
|
||||||
|
similarity: 0
|
||||||
|
}
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
const searchModeData = DatasetSearchModeMap[getValues('searchParams.searchMode')];
|
||||||
|
|
||||||
const {
|
const {
|
||||||
isOpen: isOpenSelectMode,
|
isOpen: isOpenSelectMode,
|
||||||
@ -44,14 +86,9 @@ const Test = ({ datasetId }: { datasetId: string }) => {
|
|||||||
onClose: onCloseSelectMode
|
onClose: onCloseSelectMode
|
||||||
} = useDisclosure();
|
} = useDisclosure();
|
||||||
|
|
||||||
const testHistories = useMemo(
|
const { mutate: onTextTest, isLoading: textTestIsLoading } = useRequest({
|
||||||
() => datasetTestList.filter((item) => item.datasetId === datasetId),
|
mutationFn: ({ inputText, searchParams }: FormType) =>
|
||||||
[datasetId, datasetTestList]
|
postSearchText({ datasetId, text: inputText.trim(), ...searchParams }),
|
||||||
);
|
|
||||||
|
|
||||||
const { mutate, isLoading } = useRequest({
|
|
||||||
mutationFn: () =>
|
|
||||||
postSearchText({ datasetId, text: inputText.trim(), searchMode, usingReRank, limit: 20 }),
|
|
||||||
onSuccess(res: SearchTestResponse) {
|
onSuccess(res: SearchTestResponse) {
|
||||||
if (!res || res.list.length === 0) {
|
if (!res || res.list.length === 0) {
|
||||||
return toast({
|
return toast({
|
||||||
@ -62,11 +99,14 @@ const Test = ({ datasetId }: { datasetId: string }) => {
|
|||||||
const testItem: SearchTestStoreItemType = {
|
const testItem: SearchTestStoreItemType = {
|
||||||
id: nanoid(),
|
id: nanoid(),
|
||||||
datasetId,
|
datasetId,
|
||||||
text: inputText.trim(),
|
text: getValues('inputText').trim(),
|
||||||
time: new Date(),
|
time: new Date(),
|
||||||
results: res.list,
|
results: res.list,
|
||||||
duration: res.duration,
|
duration: res.duration,
|
||||||
searchMode
|
searchMode: res.searchMode,
|
||||||
|
usingReRank: res.usingReRank,
|
||||||
|
limit: res.limit,
|
||||||
|
similarity: res.similarity
|
||||||
};
|
};
|
||||||
pushDatasetTestItem(testItem);
|
pushDatasetTestItem(testItem);
|
||||||
setDatasetTestItem(testItem);
|
setDatasetTestItem(testItem);
|
||||||
@ -78,6 +118,40 @@ const Test = ({ datasetId }: { datasetId: string }) => {
|
|||||||
});
|
});
|
||||||
}
|
}
|
||||||
});
|
});
|
||||||
|
const { mutate: onFileTest, isLoading: fileTestIsLoading } = useRequest({
|
||||||
|
mutationFn: async ({ searchParams }: FormType) => {
|
||||||
|
if (!selectFile) return Promise.reject('File is not selected');
|
||||||
|
const { data } = await readCsvContent(selectFile);
|
||||||
|
const testList = data.slice(0, 100);
|
||||||
|
const results: SearchTestResponse[] = [];
|
||||||
|
|
||||||
|
for await (const item of testList) {
|
||||||
|
try {
|
||||||
|
const result = await postSearchText({ datasetId, text: item[0].trim(), ...searchParams });
|
||||||
|
results.push(result);
|
||||||
|
} catch (error) {
|
||||||
|
await delay(500);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
return results;
|
||||||
|
},
|
||||||
|
onSuccess(res: SearchTestResponse[]) {
|
||||||
|
console.log(res);
|
||||||
|
},
|
||||||
|
onError(err) {
|
||||||
|
toast({
|
||||||
|
title: getErrText(err),
|
||||||
|
status: 'error'
|
||||||
|
});
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
const onSelectFile = async (files: File[]) => {
|
||||||
|
const file = files[0];
|
||||||
|
if (!file) return;
|
||||||
|
setSelectFile(file);
|
||||||
|
};
|
||||||
|
|
||||||
useEffect(() => {
|
useEffect(() => {
|
||||||
setDatasetTestItem(undefined);
|
setDatasetTestItem(undefined);
|
||||||
@ -85,7 +159,7 @@ const Test = ({ datasetId }: { datasetId: string }) => {
|
|||||||
|
|
||||||
return (
|
return (
|
||||||
<Box h={'100%'} display={['block', 'flex']}>
|
<Box h={'100%'} display={['block', 'flex']}>
|
||||||
{/* input */}
|
{/* left */}
|
||||||
<Box
|
<Box
|
||||||
h={['auto', '100%']}
|
h={['auto', '100%']}
|
||||||
display={['block', 'flex']}
|
display={['block', 'flex']}
|
||||||
@ -96,11 +170,39 @@ const Test = ({ datasetId }: { datasetId: string }) => {
|
|||||||
borderRight={['none', theme.borders.base]}
|
borderRight={['none', theme.borders.base]}
|
||||||
>
|
>
|
||||||
<Box border={'2px solid'} borderColor={'primary.500'} p={3} mx={4} borderRadius={'md'}>
|
<Box border={'2px solid'} borderColor={'primary.500'} p={3} mx={4} borderRadius={'md'}>
|
||||||
<Flex alignItems={'center'}>
|
{/* header */}
|
||||||
<Box fontSize={'sm'} fontWeight={'bold'} flex={1}>
|
<Flex alignItems={'center'} justifyContent={'space-between'}>
|
||||||
<MyIcon mr={2} name={'text'} w={'18px'} h={'18px'} color={'primary.600'} />
|
<MySelect
|
||||||
{t('core.dataset.test.Test Text')}
|
size={'sm'}
|
||||||
</Box>
|
w={'150px'}
|
||||||
|
list={[
|
||||||
|
{
|
||||||
|
label: (
|
||||||
|
<Flex alignItems={'center'}>
|
||||||
|
<MyIcon mr={2} name={'text'} w={'14px'} color={'primary.600'} />
|
||||||
|
<Box fontSize={'sm'} fontWeight={'bold'} flex={1}>
|
||||||
|
{t('core.dataset.test.Test Text')}
|
||||||
|
</Box>
|
||||||
|
</Flex>
|
||||||
|
),
|
||||||
|
value: 'text'
|
||||||
|
}
|
||||||
|
// {
|
||||||
|
// label: (
|
||||||
|
// <Flex alignItems={'center'}>
|
||||||
|
// <MyIcon mr={2} name={'csvImport'} w={'14px'} color={'primary.600'} />
|
||||||
|
// <Box fontSize={'sm'} fontWeight={'bold'} flex={1}>
|
||||||
|
// {t('core.dataset.test.Batch test')}
|
||||||
|
// </Box>
|
||||||
|
// </Flex>
|
||||||
|
// ),
|
||||||
|
// value: 'file'
|
||||||
|
// }
|
||||||
|
]}
|
||||||
|
value={inputType}
|
||||||
|
onchange={(e) => setInputType(e)}
|
||||||
|
/>
|
||||||
|
|
||||||
<Button
|
<Button
|
||||||
variant={'whitePrimary'}
|
variant={'whitePrimary'}
|
||||||
leftIcon={<MyIcon name={searchModeData.icon as any} w={'14px'} />}
|
leftIcon={<MyIcon name={searchModeData.icon as any} w={'14px'} />}
|
||||||
@ -110,257 +212,293 @@ const Test = ({ datasetId }: { datasetId: string }) => {
|
|||||||
{t(searchModeData.title)}
|
{t(searchModeData.title)}
|
||||||
</Button>
|
</Button>
|
||||||
</Flex>
|
</Flex>
|
||||||
<Textarea
|
|
||||||
rows={6}
|
<Box h={'180px'}>
|
||||||
resize={'none'}
|
{inputType === 'text' && (
|
||||||
variant={'unstyled'}
|
<Textarea
|
||||||
maxLength={datasetDetail.vectorModel.maxToken}
|
h={'100%'}
|
||||||
placeholder={t('core.dataset.test.Test Text Placeholder')}
|
resize={'none'}
|
||||||
defaultValue={inputText}
|
variant={'unstyled'}
|
||||||
onBlur={(e) => setInputText(e.target.value)}
|
maxLength={datasetDetail.vectorModel.maxToken}
|
||||||
/>
|
placeholder={t('core.dataset.test.Test Text Placeholder')}
|
||||||
<Flex alignItems={'center'} justifyContent={'flex-end'}>
|
{...register('inputText', {
|
||||||
<Box mx={3} color={'myGray.500'}>
|
required: true
|
||||||
{inputText.length}
|
})}
|
||||||
</Box>
|
/>
|
||||||
<Button isDisabled={inputText === ''} isLoading={isLoading} onClick={mutate}>
|
)}
|
||||||
|
{inputType === 'file' && (
|
||||||
|
<Box pt={5}>
|
||||||
|
<Flex
|
||||||
|
p={3}
|
||||||
|
borderRadius={'md'}
|
||||||
|
borderWidth={'1px'}
|
||||||
|
borderColor={'borderColor.base'}
|
||||||
|
borderStyle={'dashed'}
|
||||||
|
bg={'white'}
|
||||||
|
cursor={'pointer'}
|
||||||
|
justifyContent={'center'}
|
||||||
|
_hover={{
|
||||||
|
bg: 'primary.100',
|
||||||
|
borderColor: 'primary.500',
|
||||||
|
borderStyle: 'solid'
|
||||||
|
}}
|
||||||
|
onClick={onOpen}
|
||||||
|
>
|
||||||
|
<MyIcon mr={2} name={'csvImport'} w={'24px'} />
|
||||||
|
<Box>
|
||||||
|
{selectFile ? selectFile.name : t('core.dataset.test.Batch test Placeholder')}
|
||||||
|
</Box>
|
||||||
|
</Flex>
|
||||||
|
<Box mt={3} fontSize={'sm'}>
|
||||||
|
读取 CSV 文件第一列进行批量测试,单次最多支持 100 组数据。
|
||||||
|
<Box
|
||||||
|
as={'span'}
|
||||||
|
color={'primary.600'}
|
||||||
|
cursor={'pointer'}
|
||||||
|
onClick={() => {
|
||||||
|
fileDownload({
|
||||||
|
text: `"问题"\n"问题1"\n"问题2"\n"问题3"`,
|
||||||
|
type: 'text/csv',
|
||||||
|
filename: 'Test Template'
|
||||||
|
});
|
||||||
|
}}
|
||||||
|
>
|
||||||
|
点击下载批量测试模板
|
||||||
|
</Box>
|
||||||
|
</Box>
|
||||||
|
</Box>
|
||||||
|
)}
|
||||||
|
</Box>
|
||||||
|
|
||||||
|
<Flex justifyContent={'flex-end'}>
|
||||||
|
<Button
|
||||||
|
size={'sm'}
|
||||||
|
isLoading={textTestIsLoading || fileTestIsLoading}
|
||||||
|
isDisabled={inputType === 'file' && !selectFile}
|
||||||
|
onClick={() => {
|
||||||
|
if (inputType === 'text') {
|
||||||
|
handleSubmit((data) => onTextTest(data))();
|
||||||
|
} else {
|
||||||
|
handleSubmit((data) => onFileTest(data))();
|
||||||
|
}
|
||||||
|
}}
|
||||||
|
>
|
||||||
{t('core.dataset.test.Test')}
|
{t('core.dataset.test.Test')}
|
||||||
</Button>
|
</Button>
|
||||||
</Flex>
|
</Flex>
|
||||||
</Box>
|
</Box>
|
||||||
<Box mt={5} flex={'1 0 0'} px={4} overflow={'overlay'} display={['none', 'block']}>
|
<Box mt={5} flex={'1 0 0'} px={4} overflow={'overlay'} display={['none', 'block']}>
|
||||||
<Flex alignItems={'center'} color={'myGray.600'}>
|
<TestHistories
|
||||||
<MyIcon mr={2} name={'history'} w={'16px'} h={'16px'} />
|
datasetId={datasetId}
|
||||||
<Box fontSize={'2xl'}>{t('core.dataset.test.test history')}</Box>
|
datasetTestItem={datasetTestItem}
|
||||||
</Flex>
|
setDatasetTestItem={setDatasetTestItem}
|
||||||
<Box mt={2}>
|
/>
|
||||||
<Flex py={2} fontWeight={'bold'} borderBottom={theme.borders.sm}>
|
|
||||||
<Box w={'80px'}>{t('core.dataset.search.search mode')}</Box>
|
|
||||||
<Box flex={1}>{t('core.dataset.test.Test Text')}</Box>
|
|
||||||
<Box w={'70px'}>{t('common.Time')}</Box>
|
|
||||||
<Box w={'14px'}></Box>
|
|
||||||
</Flex>
|
|
||||||
{testHistories.map((item) => (
|
|
||||||
<Flex
|
|
||||||
key={item.id}
|
|
||||||
p={1}
|
|
||||||
alignItems={'center'}
|
|
||||||
borderBottom={theme.borders.base}
|
|
||||||
_hover={{
|
|
||||||
bg: '#f4f4f4',
|
|
||||||
'& .delete': {
|
|
||||||
display: 'block'
|
|
||||||
}
|
|
||||||
}}
|
|
||||||
cursor={'pointer'}
|
|
||||||
fontSize={'sm'}
|
|
||||||
onClick={() => setDatasetTestItem(item)}
|
|
||||||
>
|
|
||||||
<Box w={'80px'}>
|
|
||||||
{DatasetSearchModeMap[item.searchMode] ? (
|
|
||||||
<Flex alignItems={'center'}>
|
|
||||||
<MyIcon
|
|
||||||
name={DatasetSearchModeMap[item.searchMode].icon as any}
|
|
||||||
w={'12px'}
|
|
||||||
mr={'1px'}
|
|
||||||
/>
|
|
||||||
{t(DatasetSearchModeMap[item.searchMode].title)}
|
|
||||||
</Flex>
|
|
||||||
) : (
|
|
||||||
'-'
|
|
||||||
)}
|
|
||||||
</Box>
|
|
||||||
<Box flex={1} mr={2}>
|
|
||||||
{item.text}
|
|
||||||
</Box>
|
|
||||||
<Box w={'70px'}>{formatTimeToChatTime(item.time)}</Box>
|
|
||||||
<MyTooltip label={t('core.dataset.test.delete test history')}>
|
|
||||||
<Box w={'14px'} h={'14px'}>
|
|
||||||
<MyIcon
|
|
||||||
className="delete"
|
|
||||||
name={'delete'}
|
|
||||||
w={'14px'}
|
|
||||||
display={'none'}
|
|
||||||
_hover={{ color: 'red.600' }}
|
|
||||||
onClick={(e) => {
|
|
||||||
e.stopPropagation();
|
|
||||||
delDatasetTestItemById(item.id);
|
|
||||||
datasetTestItem?.id === item.id && setDatasetTestItem(undefined);
|
|
||||||
}}
|
|
||||||
/>
|
|
||||||
</Box>
|
|
||||||
</MyTooltip>
|
|
||||||
</Flex>
|
|
||||||
))}
|
|
||||||
</Box>
|
|
||||||
</Box>
|
</Box>
|
||||||
</Box>
|
</Box>
|
||||||
{/* result show */}
|
{/* result show */}
|
||||||
<Box p={4} h={['auto', '100%']} overflow={'overlay'} flex={'1 0 0'}>
|
<Box p={4} h={['auto', '100%']} overflow={'overlay'} flex={'1 0 0'}>
|
||||||
{!datasetTestItem?.results || datasetTestItem.results.length === 0 ? (
|
<TestResults datasetTestItem={datasetTestItem} />
|
||||||
<Flex
|
|
||||||
mt={[10, 0]}
|
|
||||||
h={'100%'}
|
|
||||||
flexDirection={'column'}
|
|
||||||
alignItems={'center'}
|
|
||||||
justifyContent={'center'}
|
|
||||||
>
|
|
||||||
<MyIcon name={'empty'} color={'transparent'} w={'54px'} />
|
|
||||||
<Box mt={3} color={'myGray.600'}>
|
|
||||||
{t('core.dataset.test.test result placeholder')}
|
|
||||||
</Box>
|
|
||||||
</Flex>
|
|
||||||
) : (
|
|
||||||
<>
|
|
||||||
<Flex alignItems={'center'}>
|
|
||||||
<Box fontSize={'3xl'} color={'myGray.600'}>
|
|
||||||
{t('core.dataset.test.Test Result')}
|
|
||||||
</Box>
|
|
||||||
<MyTooltip label={t('core.dataset.test.test result tip')} forceShow>
|
|
||||||
<QuestionOutlineIcon
|
|
||||||
mx={2}
|
|
||||||
color={'myGray.600'}
|
|
||||||
cursor={'pointer'}
|
|
||||||
fontSize={'lg'}
|
|
||||||
/>
|
|
||||||
</MyTooltip>
|
|
||||||
<Box>({datasetTestItem.duration})</Box>
|
|
||||||
</Flex>
|
|
||||||
<Grid
|
|
||||||
mt={1}
|
|
||||||
gridTemplateColumns={[
|
|
||||||
'repeat(1,1fr)',
|
|
||||||
'repeat(1,1fr)',
|
|
||||||
'repeat(1,1fr)',
|
|
||||||
'repeat(1,1fr)',
|
|
||||||
'repeat(2,1fr)'
|
|
||||||
]}
|
|
||||||
gridGap={4}
|
|
||||||
>
|
|
||||||
{datasetTestItem?.results.map((item, index) => (
|
|
||||||
<Box
|
|
||||||
key={item.id}
|
|
||||||
pb={2}
|
|
||||||
borderRadius={'lg'}
|
|
||||||
border={theme.borders.base}
|
|
||||||
_notLast={{ mb: 2 }}
|
|
||||||
cursor={'pointer'}
|
|
||||||
title={t('common.Edit')}
|
|
||||||
onClick={async () => {
|
|
||||||
try {
|
|
||||||
setLoading(true);
|
|
||||||
const data = await getDatasetDataItemById(item.id);
|
|
||||||
|
|
||||||
if (!data) {
|
|
||||||
throw new Error(t('core.dataset.data.data is deleted'));
|
|
||||||
}
|
|
||||||
|
|
||||||
setEditInputData({
|
|
||||||
id: data.id,
|
|
||||||
collectionId: data.collectionId,
|
|
||||||
q: data.q,
|
|
||||||
a: data.a,
|
|
||||||
indexes: data.indexes
|
|
||||||
});
|
|
||||||
} catch (err) {
|
|
||||||
toast({
|
|
||||||
status: 'warning',
|
|
||||||
title: getErrText(err)
|
|
||||||
});
|
|
||||||
}
|
|
||||||
setLoading(false);
|
|
||||||
}}
|
|
||||||
>
|
|
||||||
<Flex p={3} alignItems={'center'} color={'myGray.500'}>
|
|
||||||
<Box
|
|
||||||
border={theme.borders.base}
|
|
||||||
px={2}
|
|
||||||
fontSize={'sm'}
|
|
||||||
mr={3}
|
|
||||||
borderRadius={'md'}
|
|
||||||
>
|
|
||||||
# {index + 1}
|
|
||||||
</Box>
|
|
||||||
<RawSourceText
|
|
||||||
fontWeight={'bold'}
|
|
||||||
color={'black'}
|
|
||||||
sourceName={item.sourceName}
|
|
||||||
sourceId={item.sourceId}
|
|
||||||
canView
|
|
||||||
/>
|
|
||||||
{/* <MyIcon name={'kbTest'} w={'14px'} />
|
|
||||||
<Progress
|
|
||||||
mx={2}
|
|
||||||
flex={'1 0 0'}
|
|
||||||
value={item.score * 100}
|
|
||||||
size="sm"
|
|
||||||
borderRadius={'20px'}
|
|
||||||
colorScheme="gray"
|
|
||||||
/>
|
|
||||||
<Box>{item.score.toFixed(4)}</Box> */}
|
|
||||||
</Flex>
|
|
||||||
<Box px={2} fontSize={'xs'} color={'myGray.600'} wordBreak={'break-word'}>
|
|
||||||
<Box>{item.q}</Box>
|
|
||||||
<Box>{item.a}</Box>
|
|
||||||
</Box>
|
|
||||||
</Box>
|
|
||||||
))}
|
|
||||||
</Grid>
|
|
||||||
</>
|
|
||||||
)}
|
|
||||||
</Box>
|
</Box>
|
||||||
|
|
||||||
{!!editInputData && (
|
|
||||||
<InputDataModal
|
|
||||||
collectionId={editInputData.collectionId}
|
|
||||||
defaultValue={editInputData}
|
|
||||||
onClose={() => setEditInputData(undefined)}
|
|
||||||
onSuccess={(data) => {
|
|
||||||
if (datasetTestItem && editInputData.id) {
|
|
||||||
const newTestItem: SearchTestStoreItemType = {
|
|
||||||
...datasetTestItem,
|
|
||||||
results: datasetTestItem.results.map((item) =>
|
|
||||||
item.id === editInputData.id
|
|
||||||
? {
|
|
||||||
...item,
|
|
||||||
q: data.q || '',
|
|
||||||
a: data.a || ''
|
|
||||||
}
|
|
||||||
: item
|
|
||||||
)
|
|
||||||
};
|
|
||||||
updateDatasetItemById(newTestItem);
|
|
||||||
setDatasetTestItem(newTestItem);
|
|
||||||
}
|
|
||||||
|
|
||||||
setEditInputData(undefined);
|
|
||||||
}}
|
|
||||||
onDelete={() => {
|
|
||||||
if (datasetTestItem && editInputData.id) {
|
|
||||||
const newTestItem = {
|
|
||||||
...datasetTestItem,
|
|
||||||
results: datasetTestItem.results.filter((item) => item.id !== editInputData.id)
|
|
||||||
};
|
|
||||||
updateDatasetItemById(newTestItem);
|
|
||||||
setDatasetTestItem(newTestItem);
|
|
||||||
}
|
|
||||||
setEditInputData(undefined);
|
|
||||||
}}
|
|
||||||
/>
|
|
||||||
)}
|
|
||||||
{isOpenSelectMode && (
|
{isOpenSelectMode && (
|
||||||
<DatasetParamsModal
|
<DatasetParamsModal
|
||||||
searchMode={searchMode}
|
{...getValues('searchParams')}
|
||||||
usingReRank={usingReRank}
|
maxTokens={20000}
|
||||||
onClose={onCloseSelectMode}
|
onClose={onCloseSelectMode}
|
||||||
onSuccess={(e) => {
|
onSuccess={(e) => {
|
||||||
setSearchMode(e.searchMode);
|
setValue('searchParams', {
|
||||||
e.usingReRank !== undefined && setUsingReRank(e.usingReRank);
|
...getValues('searchParams'),
|
||||||
|
...e
|
||||||
|
});
|
||||||
|
setRefresh((state) => !state);
|
||||||
}}
|
}}
|
||||||
/>
|
/>
|
||||||
)}
|
)}
|
||||||
|
<File onSelect={onSelectFile} />
|
||||||
</Box>
|
</Box>
|
||||||
);
|
);
|
||||||
};
|
};
|
||||||
|
|
||||||
export default Test;
|
export default React.memo(Test);
|
||||||
|
|
||||||
|
const TestHistories = React.memo(function TestHistories({
|
||||||
|
datasetId,
|
||||||
|
datasetTestItem,
|
||||||
|
setDatasetTestItem
|
||||||
|
}: {
|
||||||
|
datasetId: string;
|
||||||
|
datasetTestItem?: SearchTestStoreItemType;
|
||||||
|
setDatasetTestItem: React.Dispatch<React.SetStateAction<SearchTestStoreItemType | undefined>>;
|
||||||
|
}) {
|
||||||
|
const { t } = useTranslation();
|
||||||
|
const theme = useTheme();
|
||||||
|
const { datasetTestList, delDatasetTestItemById } = useSearchTestStore();
|
||||||
|
|
||||||
|
const testHistories = useMemo(
|
||||||
|
() => datasetTestList.filter((item) => item.datasetId === datasetId),
|
||||||
|
[datasetId, datasetTestList]
|
||||||
|
);
|
||||||
|
return (
|
||||||
|
<>
|
||||||
|
<Flex alignItems={'center'} color={'myGray.600'}>
|
||||||
|
<MyIcon mr={2} name={'history'} w={'16px'} h={'16px'} />
|
||||||
|
<Box fontSize={'2xl'}>{t('core.dataset.test.test history')}</Box>
|
||||||
|
</Flex>
|
||||||
|
<Box mt={2}>
|
||||||
|
<Flex py={2} fontWeight={'bold'} borderBottom={theme.borders.sm}>
|
||||||
|
<Box flex={'0 0 80px'}>{t('core.dataset.search.search mode')}</Box>
|
||||||
|
<Box flex={1}>{t('core.dataset.test.Test Text')}</Box>
|
||||||
|
<Box flex={'0 0 70px'}>{t('common.Time')}</Box>
|
||||||
|
<Box w={'14px'}></Box>
|
||||||
|
</Flex>
|
||||||
|
{testHistories.map((item) => (
|
||||||
|
<Flex
|
||||||
|
key={item.id}
|
||||||
|
p={1}
|
||||||
|
alignItems={'center'}
|
||||||
|
borderBottom={theme.borders.base}
|
||||||
|
_hover={{
|
||||||
|
bg: '#f4f4f4',
|
||||||
|
'& .delete': {
|
||||||
|
display: 'block'
|
||||||
|
}
|
||||||
|
}}
|
||||||
|
cursor={'pointer'}
|
||||||
|
fontSize={'sm'}
|
||||||
|
onClick={() => setDatasetTestItem(item)}
|
||||||
|
>
|
||||||
|
<Box flex={'0 0 80px'}>
|
||||||
|
{DatasetSearchModeMap[item.searchMode] ? (
|
||||||
|
<Flex alignItems={'center'}>
|
||||||
|
<MyIcon
|
||||||
|
name={DatasetSearchModeMap[item.searchMode].icon as any}
|
||||||
|
w={'12px'}
|
||||||
|
mr={'1px'}
|
||||||
|
/>
|
||||||
|
{t(DatasetSearchModeMap[item.searchMode].title)}
|
||||||
|
</Flex>
|
||||||
|
) : (
|
||||||
|
'-'
|
||||||
|
)}
|
||||||
|
</Box>
|
||||||
|
<Box flex={1} mr={2} wordBreak={'break-all'}>
|
||||||
|
{item.text}
|
||||||
|
</Box>
|
||||||
|
<Box flex={'0 0 70px'}>{formatTimeToChatTime(item.time)}</Box>
|
||||||
|
<MyTooltip label={t('core.dataset.test.delete test history')}>
|
||||||
|
<Box w={'14px'} h={'14px'}>
|
||||||
|
<MyIcon
|
||||||
|
className="delete"
|
||||||
|
name={'delete'}
|
||||||
|
w={'14px'}
|
||||||
|
display={'none'}
|
||||||
|
_hover={{ color: 'red.600' }}
|
||||||
|
onClick={(e) => {
|
||||||
|
e.stopPropagation();
|
||||||
|
delDatasetTestItemById(item.id);
|
||||||
|
datasetTestItem?.id === item.id && setDatasetTestItem(undefined);
|
||||||
|
}}
|
||||||
|
/>
|
||||||
|
</Box>
|
||||||
|
</MyTooltip>
|
||||||
|
</Flex>
|
||||||
|
))}
|
||||||
|
</Box>
|
||||||
|
</>
|
||||||
|
);
|
||||||
|
});
|
||||||
|
|
||||||
|
const TestResults = React.memo(function TestResults({
|
||||||
|
datasetTestItem
|
||||||
|
}: {
|
||||||
|
datasetTestItem?: SearchTestStoreItemType;
|
||||||
|
}) {
|
||||||
|
const { t } = useTranslation();
|
||||||
|
const theme = useTheme();
|
||||||
|
|
||||||
|
return (
|
||||||
|
<>
|
||||||
|
{!datasetTestItem?.results || datasetTestItem.results.length === 0 ? (
|
||||||
|
<Flex
|
||||||
|
mt={[10, 0]}
|
||||||
|
h={'100%'}
|
||||||
|
flexDirection={'column'}
|
||||||
|
alignItems={'center'}
|
||||||
|
justifyContent={'center'}
|
||||||
|
>
|
||||||
|
<MyIcon name={'empty'} color={'transparent'} w={'54px'} />
|
||||||
|
<Box mt={3} color={'myGray.600'}>
|
||||||
|
{t('core.dataset.test.test result placeholder')}
|
||||||
|
</Box>
|
||||||
|
</Flex>
|
||||||
|
) : (
|
||||||
|
<>
|
||||||
|
<Box fontSize={'xl'} color={'myGray.600'}>
|
||||||
|
{t('core.dataset.test.Test params')}
|
||||||
|
</Box>
|
||||||
|
<TableContainer mb={3} bg={'myGray.150'} borderRadius={'md'}>
|
||||||
|
<Table>
|
||||||
|
<Thead>
|
||||||
|
<Tr>
|
||||||
|
<Th>{t('core.dataset.search.search mode')}</Th>
|
||||||
|
<Th>{t('core.dataset.search.ReRank')}</Th>
|
||||||
|
<Th>{t('core.dataset.search.Max Tokens')}</Th>
|
||||||
|
<Th>{t('core.dataset.search.Min Similarity')}</Th>
|
||||||
|
</Tr>
|
||||||
|
</Thead>
|
||||||
|
<Tbody>
|
||||||
|
<Tr>
|
||||||
|
<Td>
|
||||||
|
<Flex alignItems={'center'}>
|
||||||
|
<MyIcon
|
||||||
|
name={DatasetSearchModeMap[datasetTestItem.searchMode]?.icon as any}
|
||||||
|
w={'12px'}
|
||||||
|
mr={'1px'}
|
||||||
|
/>
|
||||||
|
{t(DatasetSearchModeMap[datasetTestItem.searchMode]?.title)}
|
||||||
|
</Flex>
|
||||||
|
</Td>
|
||||||
|
<Td>{datasetTestItem.usingReRank ? '✅' : '❌'}</Td>
|
||||||
|
<Td>{datasetTestItem.limit}</Td>
|
||||||
|
<Td>{datasetTestItem.similarity}</Td>
|
||||||
|
</Tr>
|
||||||
|
</Tbody>
|
||||||
|
</Table>
|
||||||
|
</TableContainer>
|
||||||
|
<Flex alignItems={'center'}>
|
||||||
|
<Box fontSize={'xl'} color={'myGray.600'}>
|
||||||
|
{t('core.dataset.test.Test Result')}
|
||||||
|
</Box>
|
||||||
|
<MyTooltip label={t('core.dataset.test.test result tip')} forceShow>
|
||||||
|
<QuestionOutlineIcon mx={2} color={'myGray.600'} cursor={'pointer'} fontSize={'lg'} />
|
||||||
|
</MyTooltip>
|
||||||
|
<Box>({datasetTestItem.duration})</Box>
|
||||||
|
</Flex>
|
||||||
|
<Grid
|
||||||
|
mt={1}
|
||||||
|
gridTemplateColumns={[
|
||||||
|
'repeat(1,minmax(0, 1fr))',
|
||||||
|
'repeat(1,minmax(0, 1fr))',
|
||||||
|
'repeat(1,minmax(0, 1fr))',
|
||||||
|
'repeat(1,minmax(0, 1fr))',
|
||||||
|
'repeat(2,minmax(0, 1fr))'
|
||||||
|
]}
|
||||||
|
gridGap={4}
|
||||||
|
>
|
||||||
|
{datasetTestItem?.results.map((item, index) => (
|
||||||
|
<Box
|
||||||
|
key={item.id}
|
||||||
|
p={2}
|
||||||
|
borderRadius={'sm'}
|
||||||
|
border={theme.borders.base}
|
||||||
|
_notLast={{ mb: 2 }}
|
||||||
|
>
|
||||||
|
<QuoteItem quoteItem={item} canViewSource />
|
||||||
|
</Box>
|
||||||
|
))}
|
||||||
|
</Grid>
|
||||||
|
</>
|
||||||
|
)}
|
||||||
|
</>
|
||||||
|
);
|
||||||
|
});
|
||||||
|
|||||||
@ -156,7 +156,7 @@ const Detail = ({ datasetId, currentTab }: { datasetId: string; currentTab: `${T
|
|||||||
borderRight={theme.borders.base}
|
borderRight={theme.borders.base}
|
||||||
>
|
>
|
||||||
<Flex mb={4} alignItems={'center'}>
|
<Flex mb={4} alignItems={'center'}>
|
||||||
<Avatar src={datasetDetail.avatar} w={'34px'} borderRadius={'lg'} />
|
<Avatar src={datasetDetail.avatar} w={'34px'} borderRadius={'md'} />
|
||||||
<Box ml={2}>
|
<Box ml={2}>
|
||||||
<Box fontWeight={'bold'}>{datasetDetail.name}</Box>
|
<Box fontWeight={'bold'}>{datasetDetail.name}</Box>
|
||||||
</Box>
|
</Box>
|
||||||
|
|||||||
@ -211,7 +211,7 @@ const Kb = () => {
|
|||||||
borderWidth={1.5}
|
borderWidth={1.5}
|
||||||
borderColor={dragTargetId === dataset._id ? 'primary.600' : 'borderColor.low'}
|
borderColor={dragTargetId === dataset._id ? 'primary.600' : 'borderColor.low'}
|
||||||
bg={'white'}
|
bg={'white'}
|
||||||
borderRadius={'lg'}
|
borderRadius={'md'}
|
||||||
minH={'130px'}
|
minH={'130px'}
|
||||||
position={'relative'}
|
position={'relative'}
|
||||||
data-drag-id={dataset.type === DatasetTypeEnum.folder ? dataset._id : undefined}
|
data-drag-id={dataset.type === DatasetTypeEnum.folder ? dataset._id : undefined}
|
||||||
@ -390,7 +390,7 @@ const Kb = () => {
|
|||||||
/>
|
/>
|
||||||
)}
|
)}
|
||||||
<Flex alignItems={'center'} h={'38px'}>
|
<Flex alignItems={'center'} h={'38px'}>
|
||||||
<Avatar src={dataset.avatar} borderRadius={'lg'} w={'28px'} />
|
<Avatar src={dataset.avatar} borderRadius={'md'} w={'28px'} />
|
||||||
<Box mx={3} className="textEllipsis3">
|
<Box mx={3} className="textEllipsis3">
|
||||||
{dataset.name}
|
{dataset.name}
|
||||||
</Box>
|
</Box>
|
||||||
|
|||||||
@ -60,7 +60,7 @@ const MyModules = () => {
|
|||||||
borderWidth={'1.5px'}
|
borderWidth={'1.5px'}
|
||||||
borderColor={'borderColor.low'}
|
borderColor={'borderColor.low'}
|
||||||
bg={'white'}
|
bg={'white'}
|
||||||
borderRadius={'lg'}
|
borderRadius={'md'}
|
||||||
userSelect={'none'}
|
userSelect={'none'}
|
||||||
position={'relative'}
|
position={'relative'}
|
||||||
_hover={{
|
_hover={{
|
||||||
|
|||||||
@ -18,7 +18,8 @@ export function reRankRecall({ query, inputs }: PostReRankProps) {
|
|||||||
{
|
{
|
||||||
headers: {
|
headers: {
|
||||||
Authorization: `Bearer ${model.requestAuth}`
|
Authorization: `Bearer ${model.requestAuth}`
|
||||||
}
|
},
|
||||||
|
timeout: 120000
|
||||||
}
|
}
|
||||||
)
|
)
|
||||||
.then((data) => {
|
.then((data) => {
|
||||||
|
|||||||
@ -275,25 +275,20 @@ export async function searchDatasetData(props: {
|
|||||||
const oneChunkToken = 50;
|
const oneChunkToken = 50;
|
||||||
const estimatedLen = Math.max(20, Math.ceil(maxTokens / oneChunkToken));
|
const estimatedLen = Math.max(20, Math.ceil(maxTokens / oneChunkToken));
|
||||||
|
|
||||||
// Increase search range, reduce hnsw loss. 20 ~ 100
|
|
||||||
if (searchMode === DatasetSearchModeEnum.embedding) {
|
if (searchMode === DatasetSearchModeEnum.embedding) {
|
||||||
return {
|
return {
|
||||||
embeddingLimit: Math.min(estimatedLen, 100),
|
embeddingLimit: Math.min(estimatedLen, 80),
|
||||||
fullTextLimit: 0
|
fullTextLimit: 0
|
||||||
};
|
};
|
||||||
}
|
}
|
||||||
// 50 < 2*limit < value < 100
|
|
||||||
if (searchMode === DatasetSearchModeEnum.fullTextRecall) {
|
if (searchMode === DatasetSearchModeEnum.fullTextRecall) {
|
||||||
return {
|
return {
|
||||||
embeddingLimit: 0,
|
embeddingLimit: 0,
|
||||||
fullTextLimit: Math.min(estimatedLen, 50)
|
fullTextLimit: Math.min(estimatedLen, 50)
|
||||||
};
|
};
|
||||||
}
|
}
|
||||||
// mixed
|
|
||||||
// 50 < 2*limit < embedding < 80
|
|
||||||
// 20 < limit < fullTextLimit < 40
|
|
||||||
return {
|
return {
|
||||||
embeddingLimit: Math.min(estimatedLen, 80),
|
embeddingLimit: Math.min(estimatedLen, 60),
|
||||||
fullTextLimit: Math.min(estimatedLen, 40)
|
fullTextLimit: Math.min(estimatedLen, 40)
|
||||||
};
|
};
|
||||||
};
|
};
|
||||||
@ -340,7 +335,6 @@ export async function searchDatasetData(props: {
|
|||||||
q: data.q,
|
q: data.q,
|
||||||
a: data.a,
|
a: data.a,
|
||||||
chunkIndex: data.chunkIndex,
|
chunkIndex: data.chunkIndex,
|
||||||
indexes: data.indexes,
|
|
||||||
datasetId: String(data.datasetId),
|
datasetId: String(data.datasetId),
|
||||||
collectionId: String(data.collectionId),
|
collectionId: String(data.collectionId),
|
||||||
sourceName: collection.name || '',
|
sourceName: collection.name || '',
|
||||||
@ -389,7 +383,6 @@ export async function searchDatasetData(props: {
|
|||||||
collectionId: 1,
|
collectionId: 1,
|
||||||
q: 1,
|
q: 1,
|
||||||
a: 1,
|
a: 1,
|
||||||
indexes: 1,
|
|
||||||
chunkIndex: 1
|
chunkIndex: 1
|
||||||
}
|
}
|
||||||
)
|
)
|
||||||
@ -464,6 +457,7 @@ export async function searchDatasetData(props: {
|
|||||||
|
|
||||||
return mergeResult;
|
return mergeResult;
|
||||||
} catch (error) {
|
} catch (error) {
|
||||||
|
usingReRank = false;
|
||||||
return [];
|
return [];
|
||||||
}
|
}
|
||||||
};
|
};
|
||||||
@ -553,6 +547,11 @@ export async function searchDatasetData(props: {
|
|||||||
const rrfConcat = (
|
const rrfConcat = (
|
||||||
arr: { k: number; list: SearchDataResponseItemType[] }[]
|
arr: { k: number; list: SearchDataResponseItemType[] }[]
|
||||||
): SearchDataResponseItemType[] => {
|
): SearchDataResponseItemType[] => {
|
||||||
|
arr = arr.filter((item) => item.list.length > 0);
|
||||||
|
|
||||||
|
if (arr.length === 0) return [];
|
||||||
|
if (arr.length === 1) return arr[0].list;
|
||||||
|
|
||||||
const map = new Map<string, SearchDataResponseItemType & { rrfScore: number }>();
|
const map = new Map<string, SearchDataResponseItemType & { rrfScore: number }>();
|
||||||
|
|
||||||
// rrf
|
// rrf
|
||||||
@ -643,7 +642,7 @@ export async function searchDatasetData(props: {
|
|||||||
// embedding recall and fullText recall rrf concat
|
// embedding recall and fullText recall rrf concat
|
||||||
const rrfConcatResults = rrfConcat([
|
const rrfConcatResults = rrfConcat([
|
||||||
{ k: 60, list: embeddingRecallResults },
|
{ k: 60, list: embeddingRecallResults },
|
||||||
{ k: 60, list: fullTextRecallResults },
|
{ k: 64, list: fullTextRecallResults },
|
||||||
{ k: 60, list: reRankResults }
|
{ k: 60, list: reRankResults }
|
||||||
]);
|
]);
|
||||||
|
|
||||||
@ -685,6 +684,10 @@ export async function searchDatasetData(props: {
|
|||||||
return {
|
return {
|
||||||
searchRes: filterResultsByMaxTokens(scoreFilter, maxTokens),
|
searchRes: filterResultsByMaxTokens(scoreFilter, maxTokens),
|
||||||
tokens,
|
tokens,
|
||||||
|
searchMode,
|
||||||
|
limit: maxTokens,
|
||||||
|
similarity,
|
||||||
|
usingReRank,
|
||||||
usingSimilarityFilter
|
usingSimilarityFilter
|
||||||
};
|
};
|
||||||
}
|
}
|
||||||
|
|||||||
@ -176,10 +176,12 @@ async function completions({
|
|||||||
{
|
{
|
||||||
obj: ChatRoleEnum.Human,
|
obj: ChatRoleEnum.Human,
|
||||||
value: replaceVariable(cqModel.functionPrompt || Prompt_CQJson, {
|
value: replaceVariable(cqModel.functionPrompt || Prompt_CQJson, {
|
||||||
systemPrompt,
|
systemPrompt: systemPrompt || 'null',
|
||||||
typeList: agents.map((item) => `{"${item.value}": ${item.key}}`).join('\n'),
|
typeList: agents
|
||||||
text: `${histories.map((item) => `${item.obj}:${item.value}`).join('\n')}
|
.map((item) => `{"questionType": "${item.value}", "typeId": "${item.key}"}`)
|
||||||
Human:${userChatInput}`
|
.join('\n'),
|
||||||
|
history: histories.map((item) => `${item.obj}:${item.value}`).join('\n'),
|
||||||
|
question: userChatInput
|
||||||
})
|
})
|
||||||
}
|
}
|
||||||
];
|
];
|
||||||
@ -194,7 +196,8 @@ Human:${userChatInput}`
|
|||||||
});
|
});
|
||||||
const answer = data.choices?.[0].message?.content || '';
|
const answer = data.choices?.[0].message?.content || '';
|
||||||
|
|
||||||
const id = agents.find((item) => answer.includes(item.key))?.key || '';
|
const id =
|
||||||
|
agents.find((item) => answer.includes(item.key) || answer.includes(item.value))?.key || '';
|
||||||
|
|
||||||
return {
|
return {
|
||||||
inputTokens: data.usage?.prompt_tokens || 0,
|
inputTokens: data.usage?.prompt_tokens || 0,
|
||||||
|
|||||||
@ -52,7 +52,12 @@ export async function dispatchDatasetSearch(
|
|||||||
const concatQueries = [userChatInput];
|
const concatQueries = [userChatInput];
|
||||||
|
|
||||||
// start search
|
// start search
|
||||||
const { searchRes, tokens, usingSimilarityFilter } = await searchDatasetData({
|
const {
|
||||||
|
searchRes,
|
||||||
|
tokens,
|
||||||
|
usingSimilarityFilter,
|
||||||
|
usingReRank: searchUsingReRank
|
||||||
|
} = await searchDatasetData({
|
||||||
rawQuery: userChatInput,
|
rawQuery: userChatInput,
|
||||||
queries: concatQueries,
|
queries: concatQueries,
|
||||||
model: vectorModel.model,
|
model: vectorModel.model,
|
||||||
@ -81,7 +86,7 @@ export async function dispatchDatasetSearch(
|
|||||||
similarity: usingSimilarityFilter ? similarity : undefined,
|
similarity: usingSimilarityFilter ? similarity : undefined,
|
||||||
limit,
|
limit,
|
||||||
searchMode,
|
searchMode,
|
||||||
searchUsingReRank: usingReRank
|
searchUsingReRank: searchUsingReRank
|
||||||
}
|
}
|
||||||
};
|
};
|
||||||
}
|
}
|
||||||
|
|||||||
@ -181,7 +181,7 @@ export async function dispatchModules({
|
|||||||
});
|
});
|
||||||
}
|
}
|
||||||
|
|
||||||
// get fetch params
|
// get module running params
|
||||||
const params: Record<string, any> = {};
|
const params: Record<string, any> = {};
|
||||||
module.inputs.forEach((item: any) => {
|
module.inputs.forEach((item: any) => {
|
||||||
params[item.key] = item.value;
|
params[item.key] = item.value;
|
||||||
@ -198,6 +198,7 @@ export async function dispatchModules({
|
|||||||
inputs: params
|
inputs: params
|
||||||
};
|
};
|
||||||
|
|
||||||
|
// run module
|
||||||
const dispatchRes: Record<string, any> = await (async () => {
|
const dispatchRes: Record<string, any> = await (async () => {
|
||||||
if (callbackMap[module.flowType]) {
|
if (callbackMap[module.flowType]) {
|
||||||
return callbackMap[module.flowType](dispatchData);
|
return callbackMap[module.flowType](dispatchData);
|
||||||
@ -205,10 +206,13 @@ export async function dispatchModules({
|
|||||||
return {};
|
return {};
|
||||||
})();
|
})();
|
||||||
|
|
||||||
|
// format response data. Add modulename and moduletype
|
||||||
const formatResponseData = (() => {
|
const formatResponseData = (() => {
|
||||||
if (!dispatchRes[ModuleOutputKeyEnum.responseData]) return undefined;
|
if (!dispatchRes[ModuleOutputKeyEnum.responseData]) return undefined;
|
||||||
if (Array.isArray(dispatchRes[ModuleOutputKeyEnum.responseData]))
|
if (Array.isArray(dispatchRes[ModuleOutputKeyEnum.responseData])) {
|
||||||
return dispatchRes[ModuleOutputKeyEnum.responseData];
|
return dispatchRes[ModuleOutputKeyEnum.responseData];
|
||||||
|
}
|
||||||
|
|
||||||
return {
|
return {
|
||||||
moduleName: module.name,
|
moduleName: module.name,
|
||||||
moduleType: module.flowType,
|
moduleType: module.flowType,
|
||||||
@ -216,8 +220,16 @@ export async function dispatchModules({
|
|||||||
};
|
};
|
||||||
})();
|
})();
|
||||||
|
|
||||||
|
// Pass userChatInput
|
||||||
|
const hasUserChatInputTarget = !!module.outputs.find(
|
||||||
|
(item) => item.key === ModuleOutputKeyEnum.userChatInput
|
||||||
|
)?.targets?.length;
|
||||||
|
|
||||||
return moduleOutput(module, {
|
return moduleOutput(module, {
|
||||||
[ModuleOutputKeyEnum.finish]: true,
|
[ModuleOutputKeyEnum.finish]: true,
|
||||||
|
[ModuleOutputKeyEnum.userChatInput]: hasUserChatInputTarget
|
||||||
|
? params[ModuleOutputKeyEnum.userChatInput]
|
||||||
|
: undefined,
|
||||||
...dispatchRes,
|
...dispatchRes,
|
||||||
[ModuleOutputKeyEnum.responseData]: formatResponseData
|
[ModuleOutputKeyEnum.responseData]: formatResponseData
|
||||||
});
|
});
|
||||||
|
|||||||
@ -12,6 +12,9 @@ export type SearchTestStoreItemType = {
|
|||||||
duration: string;
|
duration: string;
|
||||||
results: SearchDataResponseItemType[];
|
results: SearchDataResponseItemType[];
|
||||||
searchMode: `${DatasetSearchModeEnum}`;
|
searchMode: `${DatasetSearchModeEnum}`;
|
||||||
|
limit: number;
|
||||||
|
usingReRank: boolean;
|
||||||
|
similarity: number;
|
||||||
};
|
};
|
||||||
|
|
||||||
type State = {
|
type State = {
|
||||||
|
|||||||
@ -417,8 +417,8 @@ export const theme = extendTheme({
|
|||||||
body: 'PingFang,Noto Sans,-apple-system,BlinkMacSystemFont,"Segoe UI",Helvetica,Arial,sans-serif,"Apple Color Emoji","Segoe UI Emoji","Segoe UI Symbol"'
|
body: 'PingFang,Noto Sans,-apple-system,BlinkMacSystemFont,"Segoe UI",Helvetica,Arial,sans-serif,"Apple Color Emoji","Segoe UI Emoji","Segoe UI Symbol"'
|
||||||
},
|
},
|
||||||
fontSizes: {
|
fontSizes: {
|
||||||
xs: '10',
|
xs: '12px',
|
||||||
sm: '12px',
|
sm: '13px',
|
||||||
md: '14px',
|
md: '14px',
|
||||||
lg: '16px',
|
lg: '16px',
|
||||||
xl: '18px',
|
xl: '18px',
|
||||||
@ -440,7 +440,7 @@ export const theme = extendTheme({
|
|||||||
md: '1px solid #DAE0E2',
|
md: '1px solid #DAE0E2',
|
||||||
lg: '1px solid #D0E0E2'
|
lg: '1px solid #D0E0E2'
|
||||||
},
|
},
|
||||||
borderRadius: {
|
radii: {
|
||||||
xs: '4px',
|
xs: '4px',
|
||||||
sm: '6px',
|
sm: '6px',
|
||||||
md: '8px',
|
md: '8px',
|
||||||
|
|||||||
Loading…
x
Reference in New Issue
Block a user