feat: 检索页聚合搜索，修复中文关键词导致主进程崩溃
一次查询所有已启用数据源并按源分组展示，每组最多 5 条， 超出时提供「查看更多」跳转到该源的单源检索。 同时修复聚合并发暴露出的网络层问题： - 主进程崩溃：Memory of the World 会把搜索关键词原样回写进 ETag 响应头，中文关键词下 Electron net.fetch 在内部 emit 回调里抛 ByteString TypeError，await 无法捕获，主进程直接 崩溃且 Promise 永不 settle。现拦截该类异常并回退到 undici。 - OpenLibrary / LibGen 要求关键词至少 3 字符，否则返回 422。 改为前置校验，直接返回空结果加说明，耗时从 14s 降到 3ms。 - 重试只针对可自愈错误（超时/5xx/限流），不再为 4xx 白等； 重试时收紧超时，避免最坏耗时翻倍。镜像轮询类源关闭内层 重试，换镜像交给 tryMirrors/raceMirrors。 - 网络错误文案改为可读提示，不再把整条 URL 抛给用户。 - Sci-Hub 非 DOI 关键词返回空而非报错，避免聚合结果刷屏。 Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
This commit is contained in:
lofyer
co-authored by
factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
parent
1a1288ce18
commit
de3e1d8a44
@@ -0,0 +1,138 @@
|
||||
# PeopleLib
|
||||
|
||||
开放获取文献与图书的桌面客户端（Electron）。在一个界面里检索多个公开文献源，查看详情，下载文件并归入本地书库。
|
||||
|
||||
## 功能
|
||||
|
||||
- **多源检索**：12 个数据源统一的搜索、详情、下载流程
|
||||
- **本地书库**：收藏条目、下载文件、封面缓存、阅读状态管理
|
||||
- **全局代理**：一处配置，对所有数据源与封面请求生效
|
||||
- **镜像故障转移**：镜像失效自动切换，恢复后自动重新启用
|
||||
- **Z-Library 登录**：凭据本地保存，会话过期自动重新登录
|
||||
|
||||
## 数据源
|
||||
|
||||
| 源 | ID | 说明 |
|
||||
|---|---|---|
|
||||
| arXiv 论文 | `arxiv` | 预印本，支持全文检索 |
|
||||
| Gutenberg 公版书 | `gutenberg` | 公共领域图书 |
|
||||
| Open Library 图书 | `openlibrary` | 图书元数据与借阅入口 |
|
||||
| DOAJ 开放期刊 | `doaj` | 开放获取期刊论文 |
|
||||
| PMC 生物医学 | `pmc` | PubMed Central 全文 |
|
||||
| bioRxiv 预印本 | `biorxiv` | 仅浏览最新列表，不支持关键词搜索 |
|
||||
| Standard Ebooks | `standardebooks` | 精校排版的公版电子书 |
|
||||
| Semantic Scholar | `semanticscholar` | 学术论文检索 |
|
||||
| Memory of the World | `motw` | 公共图书馆藏书 |
|
||||
| Library Genesis | `libgen` | 图书检索，下载需站点账号 |
|
||||
| Z-Library | `zlib` | 需登录，可获取下载直链 |
|
||||
| Sci-Hub（按 DOI） | `scihub` | 按 DOI 查询 |
|
||||
|
||||
数据源的可用性取决于站点自身状态。部分站点（如 Sci-Hub）启用了人机验证，程序会给出明确提示而非静默失败。
|
||||
|
||||
## 环境要求
|
||||
|
||||
- Node.js 18+
|
||||
- Windows / macOS / Linux
|
||||
|
||||
## 快速开始
|
||||
|
||||
```bash
|
||||
npm install
|
||||
npm start
|
||||
```
|
||||
|
||||
## 打包
|
||||
|
||||
生成 Windows 免安装版到 `dist/`：
|
||||
|
||||
```bash
|
||||
npm run portable
|
||||
```
|
||||
|
||||
## 配置
|
||||
|
||||
### 代理
|
||||
|
||||
在应用内「设置」填写代理地址，例如 `http://127.0.0.1:7897`。留空表示直连。配置会持久化，重启后仍生效，并应用于所有数据源请求与封面加载。
|
||||
|
||||
网络受限环境下，多数数据源需要代理才能访问。
|
||||
|
||||
### Z-Library 账号
|
||||
|
||||
「设置」中填入邮箱与密码即可登录。登录后可获取下载直链（免费账号有每日下载额度限制）。
|
||||
|
||||
> 凭据以 base64 混淆后保存在本地 `zlib-auth.json`，**这只是防止肉眼直读，不是加密**。请勿在不受信任的机器上使用。
|
||||
|
||||
## 数据位置
|
||||
|
||||
| 模式 | 路径 |
|
||||
|---|---|
|
||||
| 开发运行 | `%APPDATA%/PeopleLib`（Windows） |
|
||||
| 打包运行 | 可执行文件同级的 `data/` 目录 |
|
||||
|
||||
该目录包含：
|
||||
|
||||
- `library.json` 书库索引
|
||||
- `settings.json` 应用设置（代理等）
|
||||
- `zlib-auth.json` Z-Library 凭据与会话
|
||||
- `covers/` 封面缓存
|
||||
|
||||
## 项目结构
|
||||
|
||||
```
|
||||
main.js 主进程：窗口、IPC、代理与证书处理、文件下载
|
||||
preload.js 渲染进程 API 桥接
|
||||
src/
|
||||
settings.js 设置持久化
|
||||
library/store.js 书库存储
|
||||
sources/
|
||||
index.js 数据源注册表
|
||||
http.js 统一 HTTP 层：超时、Cookie、代理
|
||||
mirror.js 镜像故障转移
|
||||
zlib-auth.js Z-Library 凭据存储
|
||||
<source>.js 各数据源实现
|
||||
ui/ 渲染进程界面
|
||||
```
|
||||
|
||||
### 数据源接口
|
||||
|
||||
每个数据源模块导出以下结构：
|
||||
|
||||
```js
|
||||
module.exports = {
|
||||
id: 'example',
|
||||
name: '示例源',
|
||||
supportsSearch: true,
|
||||
|
||||
async list(page) // 浏览：{ items, maxPage, page }
|
||||
async search(keyword, page) // 搜索：{ items, maxPage, page }
|
||||
async detail(postId) // 详情：{ title, authors, cover, tags, brief, links }
|
||||
async download(postId) // 下载：{ files, links }
|
||||
};
|
||||
```
|
||||
|
||||
新增数据源只需实现该接口，并在 `src/sources/index.js` 中注册。
|
||||
|
||||
### 网络层说明
|
||||
|
||||
- 主进程使用 Electron 的 `net.fetch`（Chromium 网络栈），代理通过 `session.setProxy` 生效
|
||||
- 纯 Node 环境回退到 undici `ProxyAgent`（Electron 下不启用，该组合存在兼容问题）
|
||||
- 所有请求默认 15 秒超时，可按调用点覆盖
|
||||
- 失效镜像进入 5 分钟冷却，之后自动重试，避免站点恢复后被永久跳过
|
||||
- **应用启动时全局忽略 TLS 证书错误**。这是为了兼容大量使用自签名或过期证书的镜像站点，代价是失去对中间人攻击的防护，请仅在受信任的网络环境中使用。
|
||||
|
||||
## 测试
|
||||
|
||||
```bash
|
||||
npx electron test-search.js --proxy http://127.0.0.1:7897
|
||||
```
|
||||
|
||||
对主要数据源依次执行搜索、详情、下载链路，输出每一步耗时与结果。
|
||||
|
||||
## 免责声明
|
||||
|
||||
本项目仅是对公开网络接口的客户端封装，不托管、不分发任何内容。部分数据源所提供作品的版权状态因司法辖区而异，使用者需自行确保其使用方式符合当地法律与各站点的服务条款。
|
||||
|
||||
## 许可
|
||||
|
||||
MIT
|
||||
@@ -2,6 +2,9 @@ const { fetchJson, clampPage } = require('./http');
|
||||
|
||||
const BASE = 'https://gutendex.com/books';
|
||||
|
||||
// gutendex 偶发瞬时超时，fetchJson 默认已自动重试一次
|
||||
const getJson = fetchJson;
|
||||
|
||||
function coverOf(formats) {
|
||||
if (!formats) return '';
|
||||
return formats['image/jpeg'] || formats['image/png'] || '';
|
||||
@@ -49,7 +52,7 @@ module.exports = {
|
||||
|
||||
async list(page) {
|
||||
page = clampPage(page);
|
||||
const j = await fetchJson(`${BASE}?page=${page}`);
|
||||
const j = await getJson(`${BASE}?page=${page}`);
|
||||
const total = j.count || 0;
|
||||
const maxPage = Math.max(1, Math.ceil(total / 32));
|
||||
return { items: (j.results || []).map(toItem), maxPage, page };
|
||||
@@ -57,14 +60,14 @@ module.exports = {
|
||||
|
||||
async search(keyword, page) {
|
||||
page = clampPage(page);
|
||||
const j = await fetchJson(`${BASE}?search=${encodeURIComponent(keyword)}&page=${page}`);
|
||||
const j = await getJson(`${BASE}?search=${encodeURIComponent(keyword)}&page=${page}`);
|
||||
const total = j.count || 0;
|
||||
const maxPage = Math.max(1, Math.ceil(total / 32));
|
||||
return { items: (j.results || []).map(toItem), maxPage, page };
|
||||
},
|
||||
|
||||
async detail(postId) {
|
||||
const b = await fetchJson(`${BASE}/${encodeURIComponent(postId)}`);
|
||||
const b = await getJson(`${BASE}/${encodeURIComponent(postId)}`);
|
||||
return {
|
||||
postId: String(b.id),
|
||||
title: b.title,
|
||||
@@ -79,7 +82,7 @@ module.exports = {
|
||||
},
|
||||
|
||||
async download(postId) {
|
||||
const b = await fetchJson(`${BASE}/${encodeURIComponent(postId)}`);
|
||||
const b = await getJson(`${BASE}/${encodeURIComponent(postId)}`);
|
||||
const files = bookFiles(b.formats).map((f) => ({
|
||||
name: `${String(b.title).replace(/[\\/:*?"<>|]/g, '_').slice(0, 60)}.${EXT[f.format] || 'bin'}`,
|
||||
link: f.link,
|
||||
|
||||
+154
-15
@@ -9,9 +9,8 @@ function setProxy(url) {
|
||||
proxyUrl = String(url || '').trim();
|
||||
dispatcher = null;
|
||||
if (!proxyUrl) return;
|
||||
// Electron 下由 session.setProxy 统一接管代理，不需要 undici dispatcher。
|
||||
// 仅在纯 Node 环境（脚本/测试）才构建 ProxyAgent。
|
||||
if (process.versions.electron) return;
|
||||
// Electron 下代理主要由 session.setProxy 接管；这里仍然构建 ProxyAgent，
|
||||
// 供 net.fetch 不可用时的 undici 回退路径使用。
|
||||
try {
|
||||
const { ProxyAgent } = require('undici');
|
||||
dispatcher = new ProxyAgent({
|
||||
@@ -25,18 +24,96 @@ function setProxy(url) {
|
||||
|
||||
function getProxy() { return proxyUrl; }
|
||||
|
||||
function fetchWithProxy(url, options = {}) {
|
||||
// 在 Electron 主进程中优先使用 net.fetch（走 Chromium 网络栈，由 session.setProxy 控制代理）
|
||||
if (process.versions.electron) {
|
||||
try {
|
||||
const { net } = require('electron');
|
||||
return net.fetch(url, options);
|
||||
} catch (e) { /* fallback */ }
|
||||
// --- Electron net.fetch 响应头非 ASCII 崩溃的兜底 ---
|
||||
// 部分站点（如 Memory of the World）会把搜索关键词原样回写进 ETag 等响应头。
|
||||
// 关键词含中文时，Electron 的 net.fetch 在其内部 emit 回调里抛出
|
||||
// "Cannot convert argument to a ByteString"。该异常无法被 await 捕获：
|
||||
// 主进程直接崩溃，且对应的 fetch Promise 永远不会 settle。
|
||||
//
|
||||
// 兜底策略：拦截这一类 uncaughtException（其余异常原样交还默认处理），
|
||||
// 记录 origin 后让仍在挂起中的同源请求转为可捕获的失败，再用 undici 重试。
|
||||
// 加宽限期是因为异常与请求之间无法直接关联，宽限期内正常返回的请求不受影响。
|
||||
const HEADER_BUG_GRACE = 1200;
|
||||
|
||||
// origin -> 该源已知会触发响应头崩溃（后续请求直接走 undici）
|
||||
const headerBugOrigins = new Set();
|
||||
// 正在进行中的 net.fetch: origin -> Set<abortFn>
|
||||
const pendingByOrigin = new Map();
|
||||
|
||||
function originOf(url) {
|
||||
try { return new URL(url).origin; } catch (e) { return ''; }
|
||||
}
|
||||
|
||||
function isHeaderByteStringError(e) {
|
||||
return !!e && (e instanceof TypeError || e.name === 'TypeError') && /ByteString/i.test(e.message || '');
|
||||
}
|
||||
|
||||
let rethrowing = false;
|
||||
function onUncaught(e) {
|
||||
if (isHeaderByteStringError(e)) {
|
||||
// 只有在宽限期后仍未返回的请求，才判定为被该异常卡死
|
||||
const victims = [];
|
||||
for (const set of pendingByOrigin.values()) victims.push(...set);
|
||||
setTimeout(() => { for (const fail of victims) fail(); }, HEADER_BUG_GRACE);
|
||||
return;
|
||||
}
|
||||
if (rethrowing) return;
|
||||
rethrowing = true;
|
||||
process.removeListener('uncaughtException', onUncaught);
|
||||
setImmediate(() => { throw e; });
|
||||
}
|
||||
process.on('uncaughtException', onUncaught);
|
||||
|
||||
function undiciFetch(url, options) {
|
||||
if (dispatcher) return fetch(url, { ...options, dispatcher });
|
||||
return fetch(url, options);
|
||||
}
|
||||
|
||||
function netFetch(url, options) {
|
||||
const { net } = require('electron');
|
||||
const origin = originOf(url);
|
||||
return new Promise((resolve, reject) => {
|
||||
let settled = false;
|
||||
let set = pendingByOrigin.get(origin);
|
||||
if (!set) { set = new Set(); pendingByOrigin.set(origin, set); }
|
||||
|
||||
const fail = () => {
|
||||
if (settled) return;
|
||||
settled = true;
|
||||
cleanup();
|
||||
headerBugOrigins.add(origin);
|
||||
const err = new Error('响应头包含非 ASCII 字符，Electron 网络栈无法处理');
|
||||
err.code = 'HEADER_BYTESTRING';
|
||||
reject(err);
|
||||
};
|
||||
function cleanup() {
|
||||
set.delete(fail);
|
||||
if (!set.size) pendingByOrigin.delete(origin);
|
||||
}
|
||||
set.add(fail);
|
||||
|
||||
net.fetch(url, options).then(
|
||||
(r) => { if (settled) return; settled = true; cleanup(); resolve(r); },
|
||||
(e) => { if (settled) return; settled = true; cleanup(); reject(e); }
|
||||
);
|
||||
});
|
||||
}
|
||||
|
||||
async function fetchWithProxy(url, options = {}) {
|
||||
// Electron 主进程优先用 net.fetch（走 Chromium 网络栈，由 session.setProxy 控制代理）
|
||||
if (process.versions.electron) {
|
||||
// 已知有问题的源直接走 undici，避免每次都重新触发一遍崩溃
|
||||
if (headerBugOrigins.has(originOf(url))) return undiciFetch(url, options);
|
||||
try {
|
||||
return await netFetch(url, options);
|
||||
} catch (e) {
|
||||
if (e && e.code === 'HEADER_BYTESTRING') return undiciFetch(url, options);
|
||||
throw e;
|
||||
}
|
||||
}
|
||||
return undiciFetch(url, options);
|
||||
}
|
||||
|
||||
// 简易 cookie jar: Map<domain, Map<name, value>>
|
||||
const cookieJar = new Map();
|
||||
|
||||
@@ -101,7 +178,12 @@ async function fetchRaw(url, options = {}) {
|
||||
return res;
|
||||
} catch (e) {
|
||||
if (e && (e.name === 'AbortError' || /abort/i.test(e.message || ''))) {
|
||||
throw new Error(`请求超时: ${url}`);
|
||||
throw new Error('请求超时，站点无响应');
|
||||
}
|
||||
// undici / Chromium 的底层网络错误信息很不友好，统一换成可读文案
|
||||
const msg = (e && e.message) || String(e);
|
||||
if (/fetch failed|ERR_|ENOTFOUND|ECONNREFUSED|ECONNRESET|EAI_AGAIN|socket hang up/i.test(msg)) {
|
||||
throw new Error('网络连接失败，请检查网络或代理设置');
|
||||
}
|
||||
throw e;
|
||||
} finally {
|
||||
@@ -109,18 +191,49 @@ async function fetchRaw(url, options = {}) {
|
||||
}
|
||||
}
|
||||
|
||||
async function fetchText(url, options = {}) {
|
||||
const STATUS_HINT = {
|
||||
403: '站点拒绝访问（403）',
|
||||
404: '资源不存在（404）',
|
||||
422: '查询条件不被接受（422）',
|
||||
429: '请求过于频繁，请稍后再试（429）',
|
||||
500: '站点内部错误（500）',
|
||||
502: '站点网关错误（502）',
|
||||
503: '站点暂时不可用（503）',
|
||||
504: '站点网关超时（504）'
|
||||
};
|
||||
|
||||
async function fetchOnce(url, options) {
|
||||
const res = await fetchRaw(url, options);
|
||||
if (!res.ok) throw new Error(`请求失败: ${res.status} ${url}`);
|
||||
if (!res.ok) throw new Error(STATUS_HINT[res.status] || `请求失败（HTTP ${res.status}）`);
|
||||
return res.text();
|
||||
}
|
||||
|
||||
// 默认对瞬时故障（超时/网络抖动/5xx/限流）自动重试一次。
|
||||
// 镜像轮询类数据源（LibGen / Sci-Hub / Z-Library）应传 retries: 0，
|
||||
// 由上层的 tryMirrors / raceMirrors 负责换镜像，避免重试叠加放大耗时。
|
||||
//
|
||||
// 重试会把最坏耗时翻倍（超时 15s -> 30s），聚合搜索下体感很差。
|
||||
// 因此重试时缩短单次超时：宁可放弃这次重试，也不要让整组结果一直转圈。
|
||||
function fetchText(url, options = {}) {
|
||||
const { retries, retryDelay, ...rest } = options;
|
||||
const tries = (retries === undefined ? 1 : retries) + 1;
|
||||
if (tries <= 1) return fetchOnce(url, rest);
|
||||
|
||||
const baseTimeout = rest.timeout === undefined ? DEFAULT_TIMEOUT : rest.timeout;
|
||||
return withRetry(
|
||||
(attempt) => fetchOnce(url, attempt === 0
|
||||
? rest
|
||||
: { ...rest, timeout: Math.max(3000, Math.round(baseTimeout / 2)) }),
|
||||
{ tries, delay: retryDelay || 800 }
|
||||
);
|
||||
}
|
||||
|
||||
async function fetchJson(url, options = {}) {
|
||||
const text = await fetchText(url, options);
|
||||
try {
|
||||
return JSON.parse(text);
|
||||
} catch (e) {
|
||||
throw new Error(`JSON 解析失败: ${url}`);
|
||||
throw new Error('返回内容不是有效 JSON');
|
||||
}
|
||||
}
|
||||
|
||||
@@ -145,4 +258,30 @@ function clampPage(n, min) {
|
||||
return n;
|
||||
}
|
||||
|
||||
module.exports = { UA, fetchRaw, fetchText, fetchJson, decodeEntities, stripTags, clampPage, getCookies, setCookies, clearCookies, setProxy, getProxy, fetchWithProxy };
|
||||
// 部分站点（OpenLibrary / LibGen）对过短的关键词直接返回 422。
|
||||
// 与其把 HTTP 错误抛给用户，不如提前返回空结果并带上说明。
|
||||
function tooShort(keyword, min = 3) {
|
||||
const kw = String(keyword || '').trim();
|
||||
if (kw.length >= min) return null;
|
||||
return { items: [], maxPage: 1, page: 1, note: `关键词太短，该源要求至少 ${min} 个字符` };
|
||||
}
|
||||
|
||||
// 可自愈的错误：超时、网络抖动、5xx、限流。4xx 属请求本身的问题，重试无意义。
|
||||
function isRetryable(e) {
|
||||
return /超时|网络连接失败|站点内部错误|站点网关|暂时不可用|过于频繁/.test((e && e.message) || '');
|
||||
}
|
||||
|
||||
// 带退避的重试包装。仅在错误可自愈时重试，避免为 4xx 白等几秒。
|
||||
// fn 会收到当前尝试序号（0 起），便于按次调整超时等参数。
|
||||
async function withRetry(fn, { tries = 2, delay = 1000 } = {}) {
|
||||
for (let i = 0; ; i++) {
|
||||
try {
|
||||
return await fn(i);
|
||||
} catch (e) {
|
||||
if (i >= tries - 1 || !isRetryable(e)) throw e;
|
||||
await new Promise((r) => setTimeout(r, delay * (i + 1)));
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
module.exports = { UA, fetchRaw, fetchText, fetchJson, decodeEntities, stripTags, clampPage, tooShort, isRetryable, withRetry, getCookies, setCookies, clearCookies, setProxy, getProxy, fetchWithProxy };
|
||||
|
||||
@@ -9,7 +9,7 @@
|
||||
// 策略：优先用新版站点搜索（当前唯一可用）；经典镜像作为兜底，
|
||||
// 一旦恢复即可自动参与（raceMirrors 有 5 分钟冷却重试机制）。
|
||||
|
||||
const { fetchText, decodeEntities, clampPage } = require('./http');
|
||||
const { fetchText, decodeEntities, clampPage, tooShort } = require('./http');
|
||||
const { raceMirrors } = require('./mirror');
|
||||
|
||||
// 新版站点（当前可用）
|
||||
@@ -166,7 +166,7 @@ async function webSearch(keyword, page) {
|
||||
const kw = encodeURIComponent(keyword);
|
||||
return raceMirrors('libgen-web', WEB_MIRRORS, async (base) => {
|
||||
const url = `${base}/s/${kw}${page > 1 ? `?page=${page}` : ''}`;
|
||||
const html = await fetchText(url, { timeout: TIMEOUT });
|
||||
const html = await fetchText(url, { timeout: TIMEOUT, retries: 0 });
|
||||
const items = parseWebResults(html, base);
|
||||
if (!items.length && !/searchResultBox|resItemBox|Nothing found/i.test(html)) {
|
||||
throw new Error('页面结构无法识别');
|
||||
@@ -185,7 +185,7 @@ function buildDownloadLinks(md5) {
|
||||
|
||||
async function fetchBookPage(id) {
|
||||
return raceMirrors('libgen-web', WEB_MIRRORS, async (base) => {
|
||||
const html = await fetchText(`${base}/book/${id}`, { timeout: TIMEOUT });
|
||||
const html = await fetchText(`${base}/book/${id}`, { timeout: TIMEOUT, retries: 0 });
|
||||
return { html, base };
|
||||
});
|
||||
}
|
||||
@@ -200,7 +200,7 @@ module.exports = {
|
||||
// 新版站点有 /popular 榜单
|
||||
try {
|
||||
const r = await raceMirrors('libgen-web', WEB_MIRRORS, async (base) => {
|
||||
const html = await fetchText(`${base}/popular`, { timeout: TIMEOUT });
|
||||
const html = await fetchText(`${base}/popular`, { timeout: TIMEOUT, retries: 0 });
|
||||
return { items: parseWebResults(html, base), base };
|
||||
});
|
||||
return { items: r.items, maxPage: 1, page: 1 };
|
||||
@@ -213,6 +213,8 @@ module.exports = {
|
||||
page = clampPage(page);
|
||||
const q = String(keyword || '').trim();
|
||||
if (!q) return { items: [], maxPage: 1, page };
|
||||
const short = tooShort(q);
|
||||
if (short) return { ...short, page };
|
||||
|
||||
let r;
|
||||
try {
|
||||
|
||||
@@ -1,14 +1,10 @@
|
||||
const { fetchJson, clampPage } = require('./http');
|
||||
const { fetchJson, clampPage, tooShort } = require('./http');
|
||||
|
||||
const BASE = 'https://openlibrary.org';
|
||||
const PAGE_SIZE = 20;
|
||||
|
||||
async function fetchWithRetry(url, tries = 2) {
|
||||
let last;
|
||||
for (let i = 0; i < tries; i++) {
|
||||
try { return await fetchJson(url); } catch (e) { last = e; await new Promise((r) => setTimeout(r, 1200)); }
|
||||
}
|
||||
throw last;
|
||||
function fetchWithRetry(url) {
|
||||
return fetchJson(url, { retryDelay: 1200 });
|
||||
}
|
||||
|
||||
function coverOf(doc) {
|
||||
@@ -44,6 +40,8 @@ module.exports = {
|
||||
|
||||
async search(keyword, page) {
|
||||
page = clampPage(page);
|
||||
const short = tooShort(keyword);
|
||||
if (short) return { ...short, page };
|
||||
const offset = (page - 1) * PAGE_SIZE;
|
||||
const j = await fetchWithRetry(`${BASE}/search.json?q=${encodeURIComponent(keyword)}&limit=${PAGE_SIZE}&offset=${offset}&fields=${FIELDS}`);
|
||||
const maxPage = Math.max(1, Math.ceil((j.numFound || 0) / PAGE_SIZE));
|
||||
|
||||
@@ -66,7 +66,7 @@ function extractPdf(html, base) {
|
||||
|
||||
async function fetchSciHub(base, doi) {
|
||||
const url = `${base}/${doi}`;
|
||||
const html = await fetchText(url);
|
||||
const html = await fetchText(url, { retries: 0 });
|
||||
if (isChallenge(html)) {
|
||||
throw new Error(`${base} 启用了人机验证`);
|
||||
}
|
||||
@@ -97,13 +97,12 @@ module.exports = {
|
||||
return { items: [], maxPage: 1, page: 1 };
|
||||
},
|
||||
|
||||
// 仅支持 DOI。非 DOI 关键词返回空而不是抛错，
|
||||
// 这样在聚合搜索里不会被当成"失败的源"刷屏。
|
||||
async search(keyword, page) {
|
||||
page = clampPage(page);
|
||||
const doi = normalizeDoi(keyword);
|
||||
if (!doi) return { items: [], maxPage: 1, page: 1 };
|
||||
if (!DOI_RE.test(doi)) {
|
||||
throw new Error('Sci-Hub 仅支持 DOI 查询，例如 10.1038/nature12373');
|
||||
}
|
||||
if (!doi || !DOI_RE.test(doi)) return { items: [], maxPage: 1, page: 1, note: '仅支持 DOI 查询，例如 10.1038/nature12373' };
|
||||
const r = await resolve(doi);
|
||||
return {
|
||||
items: [{
|
||||
|
||||
@@ -4,16 +4,9 @@ const BASE = 'https://api.semanticscholar.org/graph/v1';
|
||||
const PAGE_SIZE = 25;
|
||||
const FIELDS = 'title,authors,year,abstract,openAccessPdf,externalIds,url,venue';
|
||||
|
||||
async function fetchWithRetry(url, tries = 3) {
|
||||
let last;
|
||||
for (let i = 0; i < tries; i++) {
|
||||
try { return await fetchJson(url); } catch (e) {
|
||||
last = e;
|
||||
if (!/429/.test(e.message)) throw e;
|
||||
await new Promise((r) => setTimeout(r, 2500 * (i + 1)));
|
||||
}
|
||||
}
|
||||
throw last;
|
||||
// 该 API 对匿名调用限流很严。只重试一次，避免在聚合搜索里长时间阻塞整组结果。
|
||||
function fetchWithRetry(url) {
|
||||
return fetchJson(url, { retryDelay: 1500 });
|
||||
}
|
||||
|
||||
function toItem(p) {
|
||||
|
||||
+3
-1
@@ -67,6 +67,7 @@ async function doLogin() {
|
||||
const j = await fetchJson(apiUrl(m, '/eapi/user/login'), {
|
||||
method: 'POST',
|
||||
headers: FORM,
|
||||
retries: 0,
|
||||
body: form({ email: creds.email, password: creds.password })
|
||||
});
|
||||
if (!j || !j.success || !j.user) throw new Error(errMessage(j) || '登录失败');
|
||||
@@ -88,10 +89,11 @@ async function callOn(mirror, path, params, session, method) {
|
||||
j = await fetchJson(apiUrl(mirror, path), {
|
||||
method: 'POST',
|
||||
headers: FORM,
|
||||
retries: 0,
|
||||
body: form({ ...params, ...cred })
|
||||
});
|
||||
} else {
|
||||
j = await fetchJson(apiUrl(mirror, path, { ...params, ...cred }));
|
||||
j = await fetchJson(apiUrl(mirror, path, { ...params, ...cred }), { retries: 0 });
|
||||
}
|
||||
const msg = errMessage(j);
|
||||
if (msg) {
|
||||
|
||||
@@ -140,6 +140,21 @@ body {
|
||||
|
||||
.empty { grid-column: 1 / -1; text-align: center; color: var(--text-dim); padding: 60px 0; }
|
||||
|
||||
/* 聚合搜索：按源分组 */
|
||||
.agg-list { display: flex; flex-direction: column; gap: 22px; }
|
||||
.agg-list .empty { grid-column: auto; }
|
||||
.agg-group { border-top: 1px solid var(--line); padding-top: 12px; }
|
||||
.agg-group:first-child { border-top: none; padding-top: 0; }
|
||||
.agg-head { display: flex; align-items: center; gap: 10px; margin-bottom: 12px; }
|
||||
.agg-name { font-size: 14px; font-weight: 600; color: var(--text); }
|
||||
.agg-count { font-size: 12px; color: var(--text-dim); }
|
||||
.agg-count.err { color: var(--danger); }
|
||||
.agg-more { margin-left: auto; }
|
||||
.agg-grid { grid-template-columns: repeat(auto-fill, minmax(150px, 1fr)); }
|
||||
.agg-loading, .agg-msg { font-size: 12px; color: var(--text-dim); padding: 6px 0; }
|
||||
.agg-err { color: var(--danger); }
|
||||
.agg-group.agg-empty, .agg-group.agg-failed { opacity: 0.55; }
|
||||
|
||||
/* 分页 */
|
||||
.pager {
|
||||
position: sticky; bottom: -20px; z-index: 6;
|
||||
|
||||
+149
-26
@@ -1,4 +1,7 @@
|
||||
const Browse = (() => {
|
||||
const AGG_ID = '__all__';
|
||||
const AGG_LIMIT = 5;
|
||||
|
||||
const state = {
|
||||
sourceId: null,
|
||||
supportsSearch: true,
|
||||
@@ -7,6 +10,9 @@ const Browse = (() => {
|
||||
page: 1,
|
||||
maxPage: 1,
|
||||
scrollY: 0,
|
||||
sourceList: [],
|
||||
aggToken: 0,
|
||||
activeSourceId: null,
|
||||
currentPostId: null,
|
||||
currentDetail: null
|
||||
};
|
||||
@@ -34,9 +40,7 @@ const Browse = (() => {
|
||||
$('backBtn').onclick = showGrid;
|
||||
|
||||
sourceSelect.onchange = () => {
|
||||
state.sourceId = sourceSelect.value;
|
||||
state.supportsSearch = sourceSelect.selectedOptions[0].dataset.search !== '0';
|
||||
updateSearchUi();
|
||||
applySourceSelection(sourceSelect.value);
|
||||
state.mode = 'list';
|
||||
state.page = 1;
|
||||
clearSearchUi();
|
||||
@@ -46,10 +50,21 @@ const Browse = (() => {
|
||||
loadSources();
|
||||
}
|
||||
|
||||
function isAgg() { return state.sourceId === AGG_ID; }
|
||||
|
||||
function applySourceSelection(id) {
|
||||
state.sourceId = id;
|
||||
const opt = sourceSelect.selectedOptions[0];
|
||||
state.supportsSearch = id === AGG_ID ? true : (opt ? opt.dataset.search !== '0' : true);
|
||||
updateSearchUi();
|
||||
}
|
||||
|
||||
function updateSearchUi() {
|
||||
searchInput.disabled = !state.supportsSearch;
|
||||
$('searchBtn').disabled = !state.supportsSearch;
|
||||
searchInput.placeholder = state.supportsSearch ? '搜索标题 / 作者 / 关键词...' : '该源暂不支持搜索，请翻页浏览';
|
||||
searchInput.placeholder = isAgg()
|
||||
? '聚合搜索：一次查询所有已启用数据源...'
|
||||
: (state.supportsSearch ? '搜索标题 / 作者 / 关键词...' : '该源暂不支持搜索，请翻页浏览');
|
||||
}
|
||||
|
||||
async function loadSources() {
|
||||
@@ -57,7 +72,10 @@ const Browse = (() => {
|
||||
const all = res.ok ? res.data : [];
|
||||
const enabled = getEnabledSources();
|
||||
const list = enabled ? all.filter((s) => enabled.includes(s.id)) : all;
|
||||
sourceSelect.innerHTML = list.map((s) =>
|
||||
state.sourceList = list;
|
||||
const aggOpt = list.filter((s) => s.supportsSearch).length > 1
|
||||
? `<option value="${AGG_ID}" data-search="1">🔎 聚合搜索（全部源）</option>` : '';
|
||||
sourceSelect.innerHTML = aggOpt + list.map((s) =>
|
||||
`<option value="${escapeHtml(s.id)}" data-search="${s.supportsSearch ? '1' : '0'}">${escapeHtml(s.name)}</option>`).join('');
|
||||
if (!list.length) {
|
||||
state.sourceId = null;
|
||||
@@ -66,10 +84,21 @@ const Browse = (() => {
|
||||
statusBar.textContent = '';
|
||||
return;
|
||||
}
|
||||
if (!list.some((s) => s.id === state.sourceId)) state.sourceId = list[0].id;
|
||||
const stillValid = state.sourceId === AGG_ID ? !!aggOpt : list.some((s) => s.id === state.sourceId);
|
||||
if (!stillValid) state.sourceId = aggOpt ? AGG_ID : list[0].id;
|
||||
sourceSelect.value = state.sourceId;
|
||||
state.supportsSearch = sourceSelect.selectedOptions[0].dataset.search !== '0';
|
||||
updateSearchUi();
|
||||
applySourceSelection(state.sourceId);
|
||||
loadGrid();
|
||||
}
|
||||
|
||||
function gotoSource(sourceId, keyword) {
|
||||
sourceSelect.value = sourceId;
|
||||
applySourceSelection(sourceId);
|
||||
state.mode = 'search';
|
||||
state.keyword = keyword;
|
||||
state.page = 1;
|
||||
searchInput.value = keyword;
|
||||
$('clearSearchBtn').classList.remove('hidden');
|
||||
loadGrid();
|
||||
}
|
||||
|
||||
@@ -98,16 +127,21 @@ const Browse = (() => {
|
||||
}
|
||||
|
||||
async function loadGrid() {
|
||||
state.aggToken++;
|
||||
showGrid();
|
||||
statusBar.textContent = '加载中...';
|
||||
grid.innerHTML = '';
|
||||
pager.classList.add('hidden');
|
||||
mainEl.scrollTop = 0;
|
||||
|
||||
if (isAgg()) return loadAggregate();
|
||||
|
||||
const res = state.mode === 'search'
|
||||
? await window.api.sources.search(state.sourceId, state.keyword, state.page)
|
||||
: await window.api.sources.browse(state.sourceId, state.page);
|
||||
|
||||
grid.className = 'grid';
|
||||
|
||||
if (!res.ok) {
|
||||
const isAuth = state.sourceId === 'zlib' && /登录|登录|AUTH/i.test(res.error || '');
|
||||
statusBar.innerHTML = `加载失败：${escapeHtml(res.error)} <button class="retry-btn" id="gridRetry">重试</button>`;
|
||||
@@ -131,17 +165,8 @@ const Browse = (() => {
|
||||
|
||||
statusBar.textContent = state.mode === 'search' ? `搜索：${escapeHtml(state.keyword)}（第 ${state.page} 页）` : '';
|
||||
|
||||
grid.innerHTML = items.map((it) => `
|
||||
<div class="card" data-id="${escapeHtml(it.postId)}">
|
||||
<div class="card-cover" style="${coverStyle(it.cover)}">${it.cover ? '' : `<div class="ph">${escapeHtml(it.title)}</div>`}</div>
|
||||
<div class="card-title">${escapeHtml(it.title)}</div>
|
||||
${it.subtitle ? `<div class="card-sub">${escapeHtml(it.subtitle)}</div>` : ''}
|
||||
${it.date ? `<div class="card-date">${escapeHtml(it.date)}</div>` : ''}
|
||||
</div>`).join('');
|
||||
|
||||
grid.querySelectorAll('.card').forEach((el) => {
|
||||
el.onclick = () => openDetail(el.dataset.id);
|
||||
});
|
||||
grid.innerHTML = items.map(cardHtml).join('');
|
||||
bindCards(grid, state.sourceId);
|
||||
|
||||
$('pageInfo').textContent = `第 ${state.page} / ${state.maxPage} 页`;
|
||||
$('prevBtn').disabled = state.page <= 1;
|
||||
@@ -153,6 +178,97 @@ const Browse = (() => {
|
||||
pager.classList.remove('hidden');
|
||||
}
|
||||
|
||||
function cardHtml(it) {
|
||||
return `
|
||||
<div class="card" data-id="${escapeHtml(it.postId)}">
|
||||
<div class="card-cover" style="${coverStyle(it.cover)}">${it.cover ? '' : `<div class="ph">${escapeHtml(it.title)}</div>`}</div>
|
||||
<div class="card-title">${escapeHtml(it.title)}</div>
|
||||
${it.subtitle ? `<div class="card-sub">${escapeHtml(it.subtitle)}</div>` : ''}
|
||||
${it.date ? `<div class="card-date">${escapeHtml(it.date)}</div>` : ''}
|
||||
</div>`;
|
||||
}
|
||||
|
||||
function bindCards(root, sourceId) {
|
||||
root.querySelectorAll('.card').forEach((el) => {
|
||||
el.onclick = () => openDetail(el.dataset.id, sourceId);
|
||||
});
|
||||
}
|
||||
|
||||
// 聚合搜索：并发查询所有支持搜索的已启用源，按源分组展示，每组最多 AGG_LIMIT 条
|
||||
async function loadAggregate() {
|
||||
const targets = state.sourceList.filter((s) => s.supportsSearch);
|
||||
grid.className = 'agg-list';
|
||||
|
||||
if (state.mode !== 'search' || !state.keyword) {
|
||||
statusBar.textContent = '';
|
||||
grid.innerHTML = `<div class="empty">聚合搜索：输入关键词后将同时查询 ${targets.length} 个数据源</div>`;
|
||||
return;
|
||||
}
|
||||
|
||||
const token = ++state.aggToken;
|
||||
const keyword = state.keyword;
|
||||
|
||||
statusBar.textContent = `聚合搜索「${keyword}」：0 / ${targets.length} 个源完成`;
|
||||
grid.innerHTML = targets.map((s) => `
|
||||
<section class="agg-group" id="agg-${escapeHtml(s.id)}">
|
||||
<div class="agg-head">
|
||||
<span class="agg-name">${escapeHtml(s.name)}</span>
|
||||
<span class="agg-count" data-role="count">搜索中...</span>
|
||||
</div>
|
||||
<div class="grid agg-grid" data-role="grid"><div class="agg-loading">加载中...</div></div>
|
||||
</section>`).join('');
|
||||
|
||||
let done = 0;
|
||||
let hit = 0;
|
||||
let failed = 0;
|
||||
|
||||
await Promise.all(targets.map(async (s) => {
|
||||
const res = await window.api.sources.search(s.id, keyword, 1);
|
||||
if (token !== state.aggToken || !grid.isConnected) return;
|
||||
const sec = $(`agg-${s.id}`);
|
||||
if (!sec) return;
|
||||
const countEl = sec.querySelector('[data-role="count"]');
|
||||
const gridEl = sec.querySelector('[data-role="grid"]');
|
||||
const headEl = sec.querySelector('.agg-head');
|
||||
done++;
|
||||
|
||||
if (!res.ok) {
|
||||
failed++;
|
||||
sec.classList.add('agg-failed');
|
||||
countEl.textContent = '失败';
|
||||
countEl.classList.add('err');
|
||||
gridEl.className = 'agg-msg';
|
||||
gridEl.innerHTML = `<span class="agg-err">${escapeHtml(res.error)}</span>`;
|
||||
} else {
|
||||
const items = (res.data && res.data.items) || [];
|
||||
if (!items.length) {
|
||||
sec.classList.add('agg-empty');
|
||||
countEl.textContent = '无结果';
|
||||
gridEl.className = 'agg-msg';
|
||||
gridEl.textContent = res.data.note || '未找到相关结果';
|
||||
} else {
|
||||
hit++;
|
||||
const shown = items.slice(0, AGG_LIMIT);
|
||||
countEl.textContent = `${shown.length} 条`;
|
||||
gridEl.className = 'grid agg-grid';
|
||||
gridEl.innerHTML = shown.map(cardHtml).join('');
|
||||
bindCards(gridEl, s.id);
|
||||
if (items.length > AGG_LIMIT || (res.data.maxPage || 1) > 1) {
|
||||
const more = document.createElement('button');
|
||||
more.className = 'tb-btn ghost sm agg-more';
|
||||
more.textContent = '查看更多 →';
|
||||
more.onclick = () => gotoSource(s.id, keyword);
|
||||
headEl.appendChild(more);
|
||||
}
|
||||
}
|
||||
}
|
||||
statusBar.textContent = `聚合搜索「${keyword}」：${done} / ${targets.length} 个源完成`;
|
||||
}));
|
||||
|
||||
if (token !== state.aggToken) return;
|
||||
statusBar.textContent = `聚合搜索「${keyword}」：${hit} 个源有结果${failed ? `，${failed} 个源失败` : ''}（共 ${targets.length} 个源）`;
|
||||
}
|
||||
|
||||
function jumpToPage() {
|
||||
const input = $('jumpInput');
|
||||
let n = parseInt(input.value, 10);
|
||||
@@ -169,18 +285,19 @@ const Browse = (() => {
|
||||
if (state.scrollY) mainEl.scrollTop = state.scrollY;
|
||||
}
|
||||
|
||||
async function openDetail(postId) {
|
||||
async function openDetail(postId, sourceId) {
|
||||
state.scrollY = mainEl.scrollTop;
|
||||
state.currentPostId = postId;
|
||||
state.activeSourceId = sourceId || state.sourceId;
|
||||
gridView.classList.add('hidden');
|
||||
detailView.classList.remove('hidden');
|
||||
mainEl.scrollTop = 0;
|
||||
detailContent.innerHTML = '<div class="dl-loading">加载中...</div>';
|
||||
|
||||
const res = await window.api.sources.detail(state.sourceId, postId);
|
||||
const res = await window.api.sources.detail(state.activeSourceId, postId);
|
||||
if (!res.ok) {
|
||||
detailContent.innerHTML = `<div class="dl-error">加载失败：${escapeHtml(res.error)}<button class="retry-btn" id="detailRetry">重试</button></div>`;
|
||||
$('detailRetry').onclick = () => openDetail(postId);
|
||||
$('detailRetry').onclick = () => openDetail(postId, state.activeSourceId);
|
||||
return;
|
||||
}
|
||||
state.currentDetail = res.data;
|
||||
@@ -189,6 +306,11 @@ const Browse = (() => {
|
||||
refreshAddButton();
|
||||
}
|
||||
|
||||
function sourceNameOf(id) {
|
||||
const s = state.sourceList.find((x) => x.id === id);
|
||||
return s ? s.name : id;
|
||||
}
|
||||
|
||||
function renderDetail(d) {
|
||||
const tagsHtml = (d.tags || []).map((t) => {
|
||||
const idx = t.indexOf('：');
|
||||
@@ -205,6 +327,7 @@ const Browse = (() => {
|
||||
<div class="detail-meta">
|
||||
<div class="detail-title">${escapeHtml(d.title)}</div>
|
||||
${authorsHtml}
|
||||
<div class="meta-row"><b>来源</b>${escapeHtml(sourceNameOf(state.activeSourceId))}</div>
|
||||
${d.date ? `<div class="meta-row"><b>日期</b>${escapeHtml(d.date)}</div>` : ''}
|
||||
${tagsHtml}
|
||||
${d.url ? `<div class="meta-row"><b>原始链接</b><a href="#" id="detailUrlLink">${escapeHtml(d.url)}</a></div>` : ''}
|
||||
@@ -224,7 +347,7 @@ const Browse = (() => {
|
||||
async function refreshAddButton() {
|
||||
const btn = $('addLibBtn');
|
||||
if (!btn) return;
|
||||
const res = await window.api.library.findBySource(state.sourceId, state.currentPostId);
|
||||
const res = await window.api.library.findBySource(state.activeSourceId, state.currentPostId);
|
||||
if (res.ok && res.data) {
|
||||
btn.textContent = '已在书库 ✓';
|
||||
btn.disabled = true;
|
||||
@@ -242,7 +365,7 @@ const Browse = (() => {
|
||||
date: d.date || '',
|
||||
brief: d.brief || '',
|
||||
url: d.url || '',
|
||||
sourceId: state.sourceId,
|
||||
sourceId: state.activeSourceId,
|
||||
sourcePostId: state.currentPostId
|
||||
});
|
||||
if (res.ok) {
|
||||
@@ -258,7 +381,7 @@ const Browse = (() => {
|
||||
const box = $('downloadBox');
|
||||
if (!box) return;
|
||||
box.innerHTML = '<div class="dl-loading">下载信息获取中...</div>';
|
||||
const res = await window.api.sources.download(state.sourceId, postId);
|
||||
const res = await window.api.sources.download(state.activeSourceId, postId);
|
||||
if (!box.isConnected) return;
|
||||
if (!res.ok) {
|
||||
box.innerHTML = `<div class="dl-error">获取失败：${escapeHtml(res.error)}<button class="retry-btn" id="dlRetry">重试</button></div>`;
|
||||
@@ -290,7 +413,7 @@ const Browse = (() => {
|
||||
const orig = btn.textContent;
|
||||
btn.disabled = true;
|
||||
btn.textContent = '下载中...';
|
||||
const lib = await window.api.library.findBySource(state.sourceId, state.currentPostId);
|
||||
const lib = await window.api.library.findBySource(state.activeSourceId, state.currentPostId);
|
||||
const entryId = (lib.ok && lib.data) ? lib.data.id : undefined;
|
||||
const res = await window.api.downloadFile(url, btn.dataset.name || '', entryId);
|
||||
if (res.ok && res.data && res.data.canceled) {
|
||||
|
||||
Reference in New Issue
Block a user