fix(kb): 当 markitdown 不可用时，回退到纯文本解析器处理 txt/md 文件 | fall back to plain-text parser for txt/md when markitdown is unavailable - #9676
Open
SweetenedSuzuka wants to merge 2 commits into
Conversation
…AstrBotDevs#9598) Fixes AstrBotDevs#9598. Uploading .txt/.md/.markdown to a knowledge base failed with a generic "文档解析失败" error when the optional markitdown-no-magika dependency was missing, because select_parser routed plain text to MarkitdownParser and its top-level import raised ModuleNotFoundError that kb_helper masked. Modifications: - select_parser() routes .txt/.md/.markdown to the existing stdlib TextParser; .rst/.adoc/.xlsx/.docx/.xls still use MarkitdownParser - kb_helper upload_document reports a clear "缺少 markitdown-no-magika 依赖" message instead of the generic parse failure when that module is absent, and lets unrelated missing-module errors pass through - Add tests covering plain-text routing, markitdown-dependent formats, and the missing-dependency message
…rBotDevs#9598) Satisfy ruff UP012.
dosubot
Bot
added
the
size:S
![sourcery-ai[bot]](https://avatars.githubusercontent.com/in/48477?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Hey - I've left some high level feedback:
- In
select_parser, the sets of extensions routed to each parser are now hard-coded in multiple places; consider centralizing these mappings (e.g. a shared dict or constants) to keep behavior consistent and easier to update when adding new formats. - The
ModuleNotFoundErrorhandling inupload_documentrelies onexc.name == "markitdown_no_magika"; to be more robust across different raise sites, you might also guard on the module name appearing instr(exc)or centralize this dependency check into a helper function.
Prompt for AI Agents
Please address the comments from this code review:
## Overall Comments
- In `select_parser`, the sets of extensions routed to each parser are now hard-coded in multiple places; consider centralizing these mappings (e.g. a shared dict or constants) to keep behavior consistent and easier to update when adding new formats.
- The `ModuleNotFoundError` handling in `upload_document` relies on `exc.name == "markitdown_no_magika"`; to be more robust across different raise sites, you might also guard on the module name appearing in `str(exc)` or centralize this dependency check into a helper function.Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.
SweetenedSuzuka
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
摘要 / Summary
当可选依赖
markitdown-no-magika未安装时，向知识库上传.txt/.md/.markdown文件会报通用的"文档解析失败"错误，掩盖了真正的原因是缺少依赖。select_parser将纯文本格式也路由到了MarkitdownParser，该解析器在模块顶层from markitdown_no_magika import ...，依赖缺失时直接抛出ModuleNotFoundError，随后被kb_helper的except Exception统一转成"无法读取或解析上传文件"。Uploading
.txt/.md/.markdownfiles to a knowledge base failed with a generic Document parsing failed error when the optionalmarkitdown-no-magikadependency was missing, hiding the real cause.select_parserrouted plain-text formats toMarkitdownParser, whose top-levelfrom markitdown_no_magika import ...raisedModuleNotFoundErrorthatkb_helper's broadexcept Exceptionconverted into the generic "cannot read or parse the file" message.改动 / Changes*
select_parser()将.txt/.md/.markdown路由到仓库已有的标准库TextParser（支持 utf-8/gbk 等多编码解码），纯文本上传不再依赖 markitdown。.rst/.adoc/.xlsx/.docx/.xls仍使用MarkitdownParser，行为不变。kb_helper.upload_document()的解析兜底新增对ModuleNotFoundError的处理：当缺失模块正是markitdown_no_magika时，给出明确的"缺少 markitdown-no-magika 依赖"提示；其余缺失模块错误原样透传，避免误标为 markitdown 问题。select_parser()now routes.txt/.md/.markdownto the existing stdlibTextParser(multi-encoding decode: utf-8/gbk etc.), so plain-text uploads no longer depend on markitdown..rst/.adoc/.xlsx/.docx/.xlsstill useMarkitdownParser, unchanged.kb_helper.upload_document()'s parse fallback handlesModuleNotFoundError: when the missing module is exactlymarkitdown_no_magika, it surfaces a clear "missing markitdown-no-magika dependency" message; unrelated missing-module errors pass through instead of being mislabeled as a markitdown problem.验证 / Verification
新增
tests/unit/test_kb_select_parser.py，覆盖纯文本路由到TextParser、.rst/.adoc仍走MarkitdownParser、缺依赖的明确提示、无关缺失模块透传。知识库相关测试套件全部通过。
Added
tests/unit/test_kb_select_parser.pycovering plain-text routing toTextParser,.rst/.adocstill resolving toMarkitdownParser, the clear missing-dependency message, and pass-through of unrelated missing modules.The knowledge-base related test suite passes.
Fixes #9598
Summary by Sourcery
Route plain-text knowledge base uploads away from the markitdown-based parser and provide clearer error reporting when markitdown-no-magika is missing.
Bug Fixes:
Tests: