fix(scenes): 场景上传一律 400 —— diskStorage 却按内存存储判断 file.buffer
现场(真实部署 v0.7.2,用 curl 对照验证过):
POST /api/models → 201(同一份 multipart 文件)
POST /api/scenes → 400「上传场景必须带 file 字段(.glb 为主…)」
即**任何场景都传不上来**(冒烟脚本里 6 项场景断言全红)。
根因:上传中间件在 storage.module.ts 里配置为 multer 的 **diskStorage**
(先落 ${STORAGE_DIR}/tmp,避免 512 MiB 级 VRM 打爆内存),落盘后的文件对象
**只有 path、没有 buffer**;而 ScenesService.create() 当初按内存存储写:
if (!file || !file.buffer || file.buffer.length === 0) { …400… }
file.buffer 恒为 undefined,于是必然 400。
修复:与 models/motions 统一到同一套上传处理
- requireUploadedFile(file):校验确实落盘(path),并修正 busboy 按 latin1 解出的
中文/日文文件名;
- 直接导入(.glb/.gltf):storage.persistTempFile(AssetKind.Scene, upload.path, …)
—— 归档并顺带清理临时文件;
- 需转换(.zip/.pmx/.fbx):readFile(upload.path) 取 buffer 后转换 → persistBuffer,
并在 finally 中 unlink(upload.path);
- 体积上限用 upload.size 判断(UPLOAD_MAX_BYTES),超限 400 且不落盘;
- UploadedSceneFile 改为 UploadedFileLike 的别名(有 path 无 buffer);
- 日志里的文件名改用解码后的 upload.originalname。
同时全局排查过同类写法:除这两处注释外,代码里已无按 file.buffer 处理上传的地方。
新增 test/scene-upload.spec.ts(5 项,修复前必失败):diskStorage 形状的文件不再
被判成「缺少 file 字段」、GLB 直传标注未做重编码且落 pending、中文文件名被还原、
缺文件仍 400、超限 400 且不落盘。
文档:README.md/.txt 的 v0.7.3 说明补上这条;tasks/015-scene-management.md 增
「缺陷修复记录三」。
验证:npm run verify 通过(契约六项一致、35 条路由、22 文件 / 281 项测试、
nest build、启动冒烟);重新打包 opendance-web-v0.7.3.tar.gz
sha256 07463a3b9f971346d0af1f73919e6e4dfa98ef20fd160419405d57272e9d4524。
This commit is contained in:
@@ -13,6 +13,14 @@ OpenDance Studio **阶段一**: Web 管理服务端(NestJS + TypeORM + 现有 My
|
||||
**阶段一已完成**: T-WEB-001 -> T-WEB-016 全部落地,当前版本 **v0.7.3**(契约 `rest-api 0.5.0`、`websocket-keypoints 0.2.0`、`data-models 0.7.0`、`model-formats 0.5.0`、`skeleton-topology 0.2.0`、`database 0.4.0`)。
|
||||
|
||||
v0.7.3 修订(**场景表不可用不再拖垮整站** —— 生产事故修复):
|
||||
**另有一个更直接的缺陷同时修掉**:`ScenesService.create()` 当初按**内存存储**写,
|
||||
判断的是 `file.buffer`,而上传中间件(`storage.module.ts`)用的是 multer 的
|
||||
**diskStorage** —— 文件对象只有 `path`、没有 `buffer`,于是**任何场景都传不上来**,
|
||||
一律 400「上传场景必须带 file 字段」。现在与 models/motions 走同一套:
|
||||
`requireUploadedFile()` 校验落盘并修正 latin1 文件名编码,直接导入走
|
||||
`persistTempFile()`(顺带清理临时文件),需转换时 `readFile(upload.path)` 后再转,
|
||||
并在 `finally` 中清掉临时文件;
|
||||
|
||||
|
||||
1. **现场**:数据库里没有 `scenes` 表(迁移没落到该库 / 部署的 dist 里缺迁移文件),于是
|
||||
`GET /api/scenes`、`/api/catalog/export`、`/api/catalog/version`、
|
||||
|
||||
@@ -9,6 +9,11 @@ OpenDance Studio -- Web 管理服务端 v0.7.3 使用说明
|
||||
连接信息通过 .env 注入。
|
||||
|
||||
v0.7.3 修订 - 场景表不可用不再拖垮整站(生产事故修复)
|
||||
另修一个更直接的缺陷: ScenesService.create() 按内存存储写(判断 file.buffer),
|
||||
而上传中间件用的是 multer diskStorage(只有 path、没有 buffer) => 任何场景都传不上来,
|
||||
一律 400。现与 models/motions 同一套: requireUploadedFile + persistTempFile,
|
||||
需转换时 readFile(upload.path) 并在 finally 清理临时文件。
|
||||
|
||||
1. 现场: 数据库缺 scenes 表 -> /api/scenes、/api/catalog/export、
|
||||
/api/catalog/version、/api/reviews?status=pending 全部 500,
|
||||
而 /api/songs、/api/models 正常 => 客户端拉不到任何目录、管理端待审列表打不开。
|
||||
|
||||
@@ -112,3 +112,31 @@
|
||||
**测试**:`test/scenes-degradation.spec.ts`(6 项)锁定「表缺失 → 空场景而不是抛错」、
|
||||
「原因被记录并被 `/health` 暴露」、「表恢复后自动复位」、「`catalog/export` 与
|
||||
待审列表在降级下仍可用」。
|
||||
|
||||
## 缺陷修复记录三(v0.7.3):场景**根本传不上来**(diskStorage vs 内存存储)
|
||||
|
||||
**现场**:`POST /api/scenes` 一律返回 400「上传场景必须带 file 字段(.glb 为主…)」,
|
||||
而同一份 multipart 请求打 `/api/models` 返回 **201**(已经用 curl 对照验证)。
|
||||
|
||||
**根因**:上传中间件在 `storage.module.ts` 里配置为 multer 的 **`diskStorage`**
|
||||
(先落 `${STORAGE_DIR}/tmp`,避免 512 MiB 级 VRM 打爆内存),落盘后的文件对象
|
||||
**只有 `path`、没有 `buffer`**。而 `ScenesService.create()` 当初按内存存储写:
|
||||
|
||||
```ts
|
||||
if (!file || !file.buffer || file.buffer.length === 0) { …400… }
|
||||
```
|
||||
|
||||
`file.buffer` 恒为 undefined → **任何场景都传不上来**。
|
||||
|
||||
**修复**:与 models / motions 统一到同一套上传处理:
|
||||
|
||||
| 步骤 | 做法 |
|
||||
|--|--|
|
||||
| 校验 | `requireUploadedFile(file)` —— 校验确实落盘(`path` 存在),并修正 busboy 按 latin1 解出的中文/日文文件名 |
|
||||
| 直接导入(`.glb`/`.gltf`) | `storage.persistTempFile(AssetKind.Scene, upload.path, …)` —— 直接归档并**顺带清理临时文件** |
|
||||
| 需转换(`.zip`/`.pmx`/`.fbx`) | `readFile(upload.path)` 取 buffer 后转换 → `persistBuffer`,并在 `finally` 中 `unlink(upload.path)` |
|
||||
| 体积上限 | 用 `upload.size` 判断(`UPLOAD_MAX_BYTES`),超限 400 且不落盘 |
|
||||
|
||||
**测试**:`test/scene-upload.spec.ts`(5 项)—— diskStorage 形状的文件不再被判成
|
||||
"缺少 file 字段"、GLB 直传标注"未做重编码"且落 `pending`、中文文件名被还原、
|
||||
缺文件仍 400、超限 400 且不落盘。该用例在修复前必然失败。
|
||||
|
||||
@@ -1,24 +1,27 @@
|
||||
import { BadRequestException, Injectable, Logger, NotFoundException } from '@nestjs/common';
|
||||
import { InjectRepository } from '@nestjs/typeorm';
|
||||
import { existsSync } from 'node:fs';
|
||||
import { readFile, unlink } from 'node:fs/promises';
|
||||
import { Repository } from 'typeorm';
|
||||
import { AssetKind, ModelFormat, PublishStatus } from '../common/enums';
|
||||
import { assertSourceTupleComplete, type SourceTuple } from '../common/source.dto';
|
||||
import { requireUploadedFile, type UploadedFileLike } from '../common/upload.util';
|
||||
import { AppConfig } from '../config/configuration';
|
||||
import { Scene } from '../entities/scene.entity';
|
||||
import { convertModelArchive, convertModelFile } from '../models/model-convert';
|
||||
import { inspectModelFile } from '../models/model-inspector';
|
||||
import { renderModelThumbnail } from '../models/model-thumbnail';
|
||||
import { StorageService } from '../storage/storage.service';
|
||||
import type { StoredAsset } from '../storage/storage.service';
|
||||
import { normalizeSceneConfig, SCENE_CONFIG_DEFAULTS, type SceneConfig } from './scene-config';
|
||||
import { type SceneDto, toSceneDto } from './scene.serializer';
|
||||
|
||||
export interface UploadedSceneFile {
|
||||
buffer: Buffer;
|
||||
originalname: string;
|
||||
size: number;
|
||||
mimetype?: string;
|
||||
}
|
||||
/**
|
||||
* 上传文件的最小信息 —— 与 models/motions 共用 `UploadedFileLike`。
|
||||
* 注意上传中间件是 multer 的 **diskStorage**:文件只有 `path`,**没有 `buffer`**,
|
||||
* 所以不能按 `file.buffer` 判断(那会让场景永远传不上来)。
|
||||
*/
|
||||
export type UploadedSceneFile = UploadedFileLike;
|
||||
|
||||
const SUPPORTED_DIRECT = new Set(['.glb', '.gltf']);
|
||||
|
||||
@@ -96,9 +99,14 @@ export class ScenesService {
|
||||
|
||||
|
||||
async create(body: Record<string, unknown>, file: UploadedSceneFile | undefined): Promise<SceneDto> {
|
||||
if (!file || !file.buffer || file.buffer.length === 0) {
|
||||
if (!file) {
|
||||
throw new BadRequestException('上传场景必须带 file 字段(.glb 为主,也接受 .gltf/.zip/.fbx/.pmx)');
|
||||
}
|
||||
// 上传中间件用的是 **diskStorage**(见 storage.module.ts),文件只有 `path`、没有
|
||||
// `buffer`。这里必须与 models/motions 走同一套:requireUploadedFile 校验落盘 +
|
||||
// 修正 latin1 文件名编码,后续要么 persistTempFile 直接归档(顺带清理临时文件),
|
||||
// 要么读成 buffer 再转换。此前按 `file.buffer` 判断,导致**任何场景都传不上来**(400)。
|
||||
const upload = requireUploadedFile(file, 'file');
|
||||
const name = typeof body.name === 'string' && body.name.trim().length > 0 ? body.name.trim() : '';
|
||||
if (name.length === 0) throw new BadRequestException('场景名称 name 必填');
|
||||
const tuple = assertSourceTupleComplete({
|
||||
@@ -113,47 +121,63 @@ export class ScenesService {
|
||||
}
|
||||
|
||||
const parsed = this.parseConfigInput(body);
|
||||
const ext = extOf(file.originalname || '');
|
||||
let buffer: Buffer;
|
||||
const ext = extOf(upload.originalname || '');
|
||||
if (upload.size && upload.size > this.config.uploadMaxBytes) {
|
||||
throw new BadRequestException(
|
||||
`上传文件超过上限 ${Math.floor(this.config.uploadMaxBytes / 1024 / 1024)} MiB`,
|
||||
);
|
||||
}
|
||||
|
||||
let stored: StoredAsset;
|
||||
let conversionNote: string;
|
||||
let storedSize: number;
|
||||
|
||||
if (SUPPORTED_DIRECT.has(ext)) {
|
||||
// 场景主力格式:GLB 直接导入(不重编码)
|
||||
buffer = file.buffer;
|
||||
// 场景主力格式:GLB 直接导入(不重编码)——直接归档 multer 落盘的临时文件
|
||||
stored = await this.storage.persistTempFile(
|
||||
AssetKind.Scene,
|
||||
upload.path,
|
||||
upload.originalname || `scene-${Date.now()}.glb`,
|
||||
upload.mimetype || 'model/gltf-binary',
|
||||
);
|
||||
conversionNote = `直接导入(${ext.slice(1).toUpperCase()},未做重编码)`;
|
||||
storedSize = stored.asset.size;
|
||||
} else {
|
||||
// 其余格式统一转 GLB(场景不做 VRM)
|
||||
// 其余格式统一转 GLB(场景不做 VRM);转换需要 buffer,读完再转换,
|
||||
// 无论成败都要清掉临时文件(persistTempFile 那条路会自己清)
|
||||
try {
|
||||
const source = await readFile(upload.path);
|
||||
const converted =
|
||||
ext === '.zip'
|
||||
? await convertModelArchive(file.buffer, {
|
||||
? await convertModelArchive(source, {
|
||||
assimpBin: this.config.assimpBin,
|
||||
timeoutMs: this.config.modelConvertTimeoutMs,
|
||||
preferFormat: 'glb',
|
||||
})
|
||||
: await convertModelFile(file.buffer, file.originalname, {
|
||||
: await convertModelFile(source, upload.originalname, {
|
||||
assimpBin: this.config.assimpBin,
|
||||
timeoutMs: this.config.modelConvertTimeoutMs,
|
||||
preferFormat: 'glb',
|
||||
});
|
||||
buffer = converted.buffer;
|
||||
stored = await this.storage.persistBuffer(
|
||||
AssetKind.Scene,
|
||||
converted.buffer,
|
||||
upload.originalname || `scene-${Date.now()}.glb`,
|
||||
'model/gltf-binary',
|
||||
);
|
||||
storedSize = converted.buffer.length;
|
||||
conversionNote = converted.note.slice(0, 500);
|
||||
} catch (error) {
|
||||
throw new BadRequestException(
|
||||
`场景转换失败(${ext || '未知扩展名'}):${error instanceof Error ? error.message : String(error)}`,
|
||||
);
|
||||
} finally {
|
||||
await unlink(upload.path).catch(() => undefined);
|
||||
}
|
||||
}
|
||||
|
||||
const stored = await this.storage.persistBuffer(
|
||||
AssetKind.Scene,
|
||||
buffer,
|
||||
file.originalname || `scene-${Date.now()}.glb`,
|
||||
'model/gltf-binary',
|
||||
);
|
||||
|
||||
const absolutePath = this.storage.absolutePath(stored.asset);
|
||||
const stats = await this.inspectSafely(absolutePath, buffer.length, name);
|
||||
const stats = await this.inspectSafely(absolutePath, storedSize, name);
|
||||
const thumbnailAssetId = await this.thumbnailSafely(absolutePath);
|
||||
|
||||
const scene = this.scenes.create({
|
||||
@@ -175,7 +199,7 @@ export class ScenesService {
|
||||
});
|
||||
const saved = await this.scenes.save(scene);
|
||||
this.logger.log(
|
||||
`场景已入库待审:#${saved.id} ${name}(${file.originalname},${conversionNote})` +
|
||||
`场景已入库待审:#${saved.id} ${name}(${upload.originalname},${conversionNote})` +
|
||||
` 重力=(${parsed.physics.gravity.x},${parsed.physics.gravity.y},${parsed.physics.gravity.z})`,
|
||||
);
|
||||
const [dto] = await this.toDtos([saved]);
|
||||
|
||||
@@ -0,0 +1,139 @@
|
||||
import { mkdtempSync, rmSync, writeFileSync } from 'node:fs';
|
||||
import { tmpdir } from 'node:os';
|
||||
import { join } from 'node:path';
|
||||
import { afterAll, describe, expect, it, vi } from 'vitest';
|
||||
import { AssetKind, PublishStatus } from '../src/common/enums';
|
||||
import { ScenesService } from '../src/scenes/scenes.service';
|
||||
|
||||
/**
|
||||
* 场景上传必须兼容 multer 的 **diskStorage**。
|
||||
*
|
||||
* 现场(真实部署):上传中间件在 `storage.module.ts` 里配置为 `diskStorage`,
|
||||
* 落盘后的文件对象**只有 `path`、没有 `buffer`**;而 `ScenesService.create()`
|
||||
* 当初按内存存储写,判断的是 `!file.buffer`,于是**任何场景都传不上来**,
|
||||
* 一律 400「上传场景必须带 file 字段」—— 冒烟脚本里 6 项场景断言全红。
|
||||
*
|
||||
* 正确的做法与 models/motions 一致:`requireUploadedFile(file)` 校验落盘、
|
||||
* 修正 latin1 文件名编码,直接导入走 `persistTempFile()`(顺带清理临时文件),
|
||||
* 需要转换时再 `readFile(upload.path)`。
|
||||
*/
|
||||
const dir = mkdtempSync(join(tmpdir(), 'opendance-scene-upload-'));
|
||||
afterAll(() => rmSync(dir, { recursive: true, force: true }));
|
||||
|
||||
/** 一个最小的 GLB(JSON-only),仅用于让"直接导入"这条路径有文件可读 */
|
||||
function tinyGlb(): Buffer {
|
||||
const json = Buffer.from(
|
||||
JSON.stringify({
|
||||
asset: { version: '2.0' },
|
||||
scene: 0,
|
||||
scenes: [{ nodes: [0] }],
|
||||
nodes: [{ name: 'Floor_col' }],
|
||||
meshes: [{ primitives: [{ attributes: { POSITION: 0 } }] }],
|
||||
accessors: [{ count: 3 }],
|
||||
}),
|
||||
'utf8',
|
||||
);
|
||||
const padded = Buffer.concat([json, Buffer.alloc((4 - (json.length % 4)) % 4, 0x20)]);
|
||||
const header = Buffer.alloc(12);
|
||||
header.write('glTF', 0, 'ascii');
|
||||
header.writeUInt32LE(2, 4);
|
||||
header.writeUInt32LE(12 + 8 + padded.length, 8);
|
||||
const chunkHeader = Buffer.alloc(8);
|
||||
chunkHeader.writeUInt32LE(padded.length, 0);
|
||||
chunkHeader.writeUInt32LE(0x4e4f534a, 4);
|
||||
return Buffer.concat([header, chunkHeader, padded]);
|
||||
}
|
||||
|
||||
/** 造一个 **diskStorage 形状**的上传文件:有 path,没有 buffer */
|
||||
function diskUpload(name = 'stage.glb') {
|
||||
const path = join(dir, `${Date.now()}-${Math.random().toString(16).slice(2)}${name.slice(name.lastIndexOf('.'))}`);
|
||||
writeFileSync(path, tinyGlb());
|
||||
const size = tinyGlb().length;
|
||||
return { originalname: name, mimetype: 'model/gltf-binary', path, size };
|
||||
}
|
||||
|
||||
const SOURCE = {
|
||||
sourceType: 'original',
|
||||
sourceName: '测试来源',
|
||||
sourceUrl: 'https://example.com/s',
|
||||
license: 'MIT',
|
||||
author: 'QA',
|
||||
};
|
||||
|
||||
function buildService() {
|
||||
const persisted = {
|
||||
id: 42,
|
||||
kind: AssetKind.Scene,
|
||||
sha256: 'a'.repeat(64),
|
||||
storedPath: 'scene/2026/09/aa/x.glb',
|
||||
size: 216,
|
||||
};
|
||||
const persistTempFile = vi.fn(async () => ({ asset: persisted, url: '/static/scene/x.glb' }));
|
||||
const persistBuffer = vi.fn(async () => ({ asset: persisted, url: '/static/scene/x.glb' }));
|
||||
const storage = {
|
||||
persistTempFile,
|
||||
persistBuffer,
|
||||
absolutePath: vi.fn(() => join(dir, 'not-a-real-file.glb')),
|
||||
url: vi.fn(() => '/static/scene/x.glb'),
|
||||
findMany: vi.fn(async () => new Map()),
|
||||
};
|
||||
const scenesRepository = {
|
||||
create: vi.fn((value: Record<string, unknown>) => value),
|
||||
save: vi.fn(async (value: Record<string, unknown>) => ({
|
||||
...value,
|
||||
id: 7,
|
||||
createdAt: new Date(),
|
||||
updatedAt: new Date(),
|
||||
deletedAt: null,
|
||||
})),
|
||||
find: vi.fn(async () => []),
|
||||
};
|
||||
const config = { uploadMaxBytes: 512 * 1024 * 1024, assimpBin: 'assimp', modelConvertTimeoutMs: 1000 };
|
||||
const service = new ScenesService(scenesRepository as never, storage as never, config as never);
|
||||
return { service, storage, persistTempFile, persistBuffer, scenesRepository };
|
||||
}
|
||||
|
||||
describe('场景上传兼容 multer diskStorage', () => {
|
||||
it('diskStorage 形状的文件(只有 path)不会再被判成"缺少 file 字段"', async () => {
|
||||
const { service, persistTempFile } = buildService();
|
||||
const dto = await service.create({ name: '冒烟场景', ...SOURCE }, diskUpload() as never);
|
||||
|
||||
expect(dto).toBeTruthy();
|
||||
// 直接导入走 persistTempFile(落盘的临时文件被归档)
|
||||
expect(persistTempFile).toHaveBeenCalledTimes(1);
|
||||
const call = persistTempFile.mock.calls[0] as unknown[];
|
||||
expect(call[0]).toBe(AssetKind.Scene);
|
||||
expect(String(call[1])).toMatch(/\.glb$/);
|
||||
expect(call[2]).toBe('stage.glb');
|
||||
});
|
||||
|
||||
it('GLB 直传仍标注"未做重编码",并落 pending 待审', async () => {
|
||||
const { service } = buildService();
|
||||
const dto = await service.create({ name: '冒烟场景', ...SOURCE }, diskUpload() as never);
|
||||
expect(String(dto.conversionNote)).toContain('直接导入');
|
||||
expect(dto.status).toBe(PublishStatus.Pending);
|
||||
});
|
||||
|
||||
it('文件名用 latin1 乱码传入时会被还原(与 models/motions 同一套)', async () => {
|
||||
const { service, persistTempFile } = buildService();
|
||||
// busboy 会把 UTF-8 文件名按 latin1 解出来
|
||||
const mangled = Buffer.from('舞台.glb', 'utf8').toString('latin1');
|
||||
await service.create({ name: '中文名场景', ...SOURCE }, diskUpload(mangled) as never);
|
||||
const call = persistTempFile.mock.calls[0] as unknown[];
|
||||
expect(call[2]).toBe('舞台.glb');
|
||||
});
|
||||
|
||||
it('缺少文件时仍然 400(错误语义没有被削弱)', async () => {
|
||||
const { service } = buildService();
|
||||
await expect(service.create({ name: '无文件', ...SOURCE }, undefined)).rejects.toThrow(/必须带 file 字段/);
|
||||
});
|
||||
|
||||
it('超过 UPLOAD_MAX_BYTES 时 400 且不落盘', async () => {
|
||||
const { service, persistTempFile } = buildService();
|
||||
const upload = diskUpload();
|
||||
await expect(
|
||||
service.create({ name: '超大场景', ...SOURCE }, { ...upload, size: 999 * 1024 * 1024 } as never),
|
||||
).rejects.toThrow(/超过上限/);
|
||||
expect(persistTempFile).not.toHaveBeenCalled();
|
||||
});
|
||||
});
|
||||
Reference in New Issue
Block a user