mirror of
https://github.com/github/awesome-copilot.git
synced 2026-07-16 10:53:25 +00:00
feat: add convert-excel-to-md, convert-pdf-to-md, and convert-word-to-md skills (#2294)
* feat: add convert-excel-to-md, convert-pdf-to-md, and convert-word-to-md skills Add three new agent skills that convert common document formats to Markdown using bundled Python scripts powered by MarkItDown: - convert-excel-to-md: Converts .xlsx workbooks to Markdown with per-sheet tables and embedded image extraction via a bundled Python script. - convert-pdf-to-md: Converts .pdf documents to Markdown with text/table extraction and embedded image extraction via PyMuPDF. - convert-word-to-md: Converts .docx documents to Markdown with proper image extraction replacing MarkItDown's base64 placeholders. Each skill includes: - SKILL.md with detailed usage instructions, output structure docs, and a troubleshooting table - scripts/ with the conversion Python script and requirements.txt - references/setup.md with environment setup instructions Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix: remove Markdown image syntax from convert-word-to-md SKILL.md The CI valid-refs linter flagged the literal Markdown image syntax containing a data URI as an invalid file reference. Replaced it with a plain text description of the placeholder format. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix: update pip install command to use scripts/requirements.txt for setup * fix: clarify installation instructions for requirements-file in setup documentation * fix: correct indentation in image extraction function * Potential fix for pull request finding Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * feat: add comprehensive documentation and setup instructions for convert-to-md skills * fix: add trailing slashes to skill paths in plugin.json * fix: reorder skills in plugin.json for consistency * fix: update plugin.json and README.md for clarity and consistency * feat: add convert-to-md plugin and update related documentation * Potential fix for pull request finding Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * fix: enhance conversion skills to handle mixed file types and improve error handling --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
This commit is contained in:
Willie Yao
GitHub
@@ -253,6 +253,12 @@
|
||||
"description": "Coding agents hallucinate APIs. ContextMatic gives them curated, versioned API and SDK docs. Ask your agent to \"integrate the payments API\" and it guesses — falling back on outdated training data and generic patterns that don't match your actual SDK. ContextMatic solves this by giving the agent deterministic, version-aware, SDK-native context at the exact moment it's needed.",
|
||||
"version": "0.1.0"
|
||||
},
|
||||
{
|
||||
"name": "convert-to-md",
|
||||
"source": "plugins/convert-to-md",
|
||||
"description": "A collection of Copilot skills that convert common document formats into Markdown so their contents can be accurately analyzed, summarized, searched, or extracted from. Just tell Copilot what you need — the right skill is invoked automatically and the conversion happens behind the scenes.",
|
||||
"version": "1.0.1"
|
||||
},
|
||||
{
|
||||
"name": "copilot-goal-skill",
|
||||
"description": "Goal-driven task orchestration with independent verification. Interviews the user to define a clear goal, then loops between a Builder subagent (does the work) and an Inspector subagent (judges the result with fresh context). The Inspector never trusts the Builder. Output is auditable in git commits from each subagent actions. Use when the user says \"achieve this goal\", \"make this work\", \"implement until done\", or wants verified autonomous task completion with independent quality review.",
|
||||
|
||||
@@ -40,6 +40,7 @@ See [CONTRIBUTING.md](../CONTRIBUTING.md#adding-plugins) for guidelines on how t
|
||||
| [cms-development](../plugins/cms-development/README.md) | Skills for CMS development across themes, plugins, admin tooling, media workflows, markdown rendering, and static export pipelines. | 3 items | cms, content-management-system, wordpress, shopify, drupal, theme, plugin, media, static-site |
|
||||
| [context-engineering](../plugins/context-engineering/README.md) | Tools and techniques for maximizing GitHub Copilot effectiveness through better context management. Includes guidelines for structuring code, an agent for planning multi-file changes, and prompts for context-aware development. | 4 items | context, productivity, refactoring, best-practices, architecture |
|
||||
| [context-matic](../plugins/context-matic/README.md) | Coding agents hallucinate APIs. ContextMatic gives them curated, versioned API and SDK docs. Ask your agent to "integrate the payments API" and it guesses — falling back on outdated training data and generic patterns that don't match your actual SDK. ContextMatic solves this by giving the agent deterministic, version-aware, SDK-native context at the exact moment it's needed. | 2 items | api-context, api-integration, mcp, sdk, apimatic, third-party-apis, sdks |
|
||||
| [convert-to-md](../plugins/convert-to-md/README.md) | A collection of Copilot skills that convert common document formats into Markdown so their contents can be accurately analyzed, summarized, searched, or extracted from. Just tell Copilot what you need — the right skill is invoked automatically and the conversion happens behind the scenes. | 3 items | skills, configuration, copilot, convert-word-to-md, convert-excel-to-md, convert-pdf-to-md |
|
||||
| [copilot-sdk](../plugins/copilot-sdk/README.md) | Build applications with the GitHub Copilot SDK across multiple programming languages. Includes comprehensive instructions for C#, Go, Node.js/TypeScript, and Python to help you create AI-powered applications. | 1 items | copilot-sdk, sdk, csharp, go, nodejs, typescript, python, ai, github-copilot |
|
||||
| [csharp-dotnet-development](../plugins/csharp-dotnet-development/README.md) | Essential prompts, instructions, and chat modes for C# and .NET development including testing, documentation, and best practices. | 9 items | csharp, dotnet, aspnet, testing |
|
||||
| [database-data-management](../plugins/database-data-management/README.md) | Database administration, SQL optimization, and data management tools for PostgreSQL, SQL Server, and general database development best practices. | 6 items | database, sql, postgresql, sql-server, dba, optimization, queries, data-management |
|
||||
|
||||
@@ -100,7 +100,10 @@ See [CONTRIBUTING.md](../CONTRIBUTING.md#adding-skills) for guidelines on how to
|
||||
| [context-map](../skills/context-map/SKILL.md)<br />`gh skills install github/awesome-copilot context-map` | Generate a map of all files relevant to a task before making changes | None |
|
||||
| [conventional-branch](../skills/conventional-branch/SKILL.md)<br />`gh skills install github/awesome-copilot conventional-branch` | Create Git branches following the Conventional Branch specification (feature/, bugfix/, hotfix/, release/, chore/). Use when creating a new branch, naming a branch, or checking whether a branch name complies with the spec. | None |
|
||||
| [conventional-commit](../skills/conventional-commit/SKILL.md)<br />`gh skills install github/awesome-copilot conventional-commit` | Prompt and workflow for generating conventional commit messages using a structured XML format. Guides users to create standardized, descriptive commit messages in line with the Conventional Commits specification, including instructions, examples, and validation. | None |
|
||||
| [convert-excel-to-md](../skills/convert-excel-to-md/SKILL.md)<br />`gh skills install github/awesome-copilot convert-excel-to-md` | Converts Excel (.xlsx) workbooks into Markdown so their contents can be accurately analyzed, summarized, searched, or extracted from. Use this skill whenever the user shares, references, or asks about a .xlsx file — even if they don't say "convert" or "markdown" explicitly. This includes requests to "read", "summarize", "review", "extract data from", "compare", "chart", or "analyze" a spreadsheet, workbook, budget, data export, or tracker. Always run the bundled conversion script to produce Markdown first; do not attempt to parse .xlsx content directly or write ad-hoc extraction code. Also use this skill for batch requests involving a whole folder of Excel workbooks. IMPORTANT: When the user references a folder or set of documents containing multiple file types (.pdf, .docx, .xlsx), invoke ALL three sibling skills — convert-pdf-to-md, convert-word-to-md, and convert-excel-to-md — so no file type is silently skipped. | `references/setup.md`<br />`scripts/convert_excel_to_md.py`<br />`scripts/requirements.txt` |
|
||||
| [convert-pdf-to-md](../skills/convert-pdf-to-md/SKILL.md)<br />`gh skills install github/awesome-copilot convert-pdf-to-md` | Converts PDF (.pdf) documents into Markdown so their contents can be accurately analyzed, summarized, searched, or extracted from. Use this skill whenever the user shares, references, or asks about a .pdf file — even if they don't say "convert" or "markdown" explicitly. This includes requests to "read", "summarize", "review", "extract data from", "compare", or "analyze" a PDF report, paper, invoice, form, contract, or scanned document. Always run the bundled conversion script to produce Markdown first; do not attempt to parse PDF content directly or write ad-hoc extraction code. Also use this skill for batch requests involving a whole folder of PDF documents. IMPORTANT: When the user references a folder or set of documents containing multiple file types (.pdf, .docx, .xlsx), invoke ALL three sibling skills — convert-pdf-to-md, convert-word-to-md, and convert-excel-to-md — so no file type is silently skipped. | `references/setup.md`<br />`scripts/convert_pdf_to_md.py`<br />`scripts/requirements.txt` |
|
||||
| [convert-plaintext-to-md](../skills/convert-plaintext-to-md/SKILL.md)<br />`gh skills install github/awesome-copilot convert-plaintext-to-md` | Convert a text-based document to markdown following instructions from prompt, or if a documented option is passed, follow the instructions for that option. | None |
|
||||
| [convert-word-to-md](../skills/convert-word-to-md/SKILL.md)<br />`gh skills install github/awesome-copilot convert-word-to-md` | Converts Word (.docx) documents into Markdown so their contents can be accurately analyzed, summarized, searched, or extracted from. Use this skill whenever the user shares, references, or asks about a .docx file — even if they don't say "convert" or "markdown" explicitly. This includes requests to "read", "summarize", "review", "extract data from", "compare", or "analyze" a Word document, resume, report, contract, or proposal. Always run the bundled conversion script to produce Markdown first; do not attempt to parse .docx content directly or write ad-hoc conversion code. Also use this skill for batch requests involving a whole folder of Word documents. IMPORTANT: When the user references a folder or set of documents containing multiple file types (.pdf, .docx, .xlsx), invoke ALL three sibling skills — convert-pdf-to-md, convert-word-to-md, and convert-excel-to-md — so no file type is silently skipped. | `references/setup.md`<br />`scripts/convert_word_to_md.py`<br />`scripts/requirements.txt` |
|
||||
| [copilot-cli-quickstart](../skills/copilot-cli-quickstart/SKILL.md)<br />`gh skills install github/awesome-copilot copilot-cli-quickstart` | Use this skill when someone wants to learn GitHub Copilot CLI from scratch. Offers interactive step-by-step tutorials with separate Developer and Non-Developer tracks, plus on-demand Q&A. Just say "start tutorial" or ask a question! Note: This skill targets GitHub Copilot CLI specifically and uses CLI-specific tools (ask_user, sql, fetch_copilot_cli_documentation). | None |
|
||||
| [copilot-instructions-blueprint-generator](../skills/copilot-instructions-blueprint-generator/SKILL.md)<br />`gh skills install github/awesome-copilot copilot-instructions-blueprint-generator` | Technology-agnostic blueprint generator for creating comprehensive copilot-instructions.md files that guide GitHub Copilot to produce code consistent with project standards, architecture patterns, and exact technology versions by analyzing existing codebase patterns and avoiding assumptions. | None |
|
||||
| [copilot-pr-autopilot](../skills/copilot-pr-autopilot/SKILL.md)<br />`gh skills install github/awesome-copilot copilot-pr-autopilot` | Copilot left 14 review comments on your PR — half are nits. Hours of fix → reply → resolve → re-request, and each round lands MORE comments. This skill runs loop engineering: auto-triggers Copilot Code Review via GraphQL (no @copilot mention), triages every open thread (Copilot, humans, advanced-security) with a fix / decline / escalate rubric, dispatches parallel fix sub-agents that obey the repo build/test/lint conventions, commits per iteration, replies+resolves citing the pushed SHA, then re-triggers until HEAD is reviewed with zero threads awaiting the agent's reply (remaining open threads are explicit hand-offs to the human — escalated declines, design tradeoffs). You merge a clean PR; the bot runs it. Trigger phrases: "address copilot comments", "run a copilot review loop", "fix this PR", "iterate on copilot feedback". Repo-agnostic, gh CLI + PowerShell. Full autopilot needs repo Triage/Write; external PR authors get single-iteration mode plus manual re-trigger (UI 🔄 or substantive-commit push). | `references/01-request-review.md`<br />`references/02-wait.md`<br />`references/03-list-threads.md`<br />`references/04-triage.md`<br />`references/05-fix.md`<br />`references/06-build-test.md`<br />`references/07-commit-push.md`<br />`references/08-reply-resolve.md`<br />`references/09-convergence.md`<br />`references/10-cleanup.md`<br />`references/api-quirks.md`<br />`references/orchestration.md`<br />`scripts/01-request-review.ps1`<br />`scripts/02-check-review-status.ps1`<br />`scripts/03-list-open-threads.ps1`<br />`scripts/08-reply-and-resolve.ps1`<br />`scripts/09-review-round.ps1`<br />`scripts/10-cleanup-outdated.ps1`<br />`scripts/_lib.ps1`<br />`templates` |
|
||||
|
||||
@@ -0,0 +1,24 @@
|
||||
{
|
||||
"name": "convert-to-md",
|
||||
"description": "A collection of Copilot skills that convert common document formats into Markdown so their contents can be accurately analyzed, summarized, searched, or extracted from. Just tell Copilot what you need — the right skill is invoked automatically and the conversion happens behind the scenes.",
|
||||
"version": "1.0.1",
|
||||
"keywords": [
|
||||
"skills",
|
||||
"configuration",
|
||||
"copilot",
|
||||
"convert-word-to-md",
|
||||
"convert-excel-to-md",
|
||||
"convert-pdf-to-md"
|
||||
],
|
||||
"author": {
|
||||
"name": "Willie Yao",
|
||||
"url": "https://github.com/1YaoWei0"
|
||||
},
|
||||
"repository": "https://github.com/github/awesome-copilot",
|
||||
"license": "MIT",
|
||||
"skills": [
|
||||
"./skills/convert-excel-to-md/",
|
||||
"./skills/convert-pdf-to-md/",
|
||||
"./skills/convert-word-to-md/"
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,47 @@
|
||||
# Convert to Markdown Plugin
|
||||
|
||||
A collection of Copilot skills that convert common document formats into Markdown so their contents can be accurately analyzed, summarized, searched, or extracted from. Just tell Copilot what you need — the right skill is invoked automatically and the conversion happens behind the scenes.
|
||||
|
||||
## Installation
|
||||
|
||||
```bash
|
||||
copilot plugin install convert-to-md@awesome-copilot
|
||||
```
|
||||
|
||||
## What's Included
|
||||
|
||||
This plugin includes Word, Excel, and PDF conversion skills, detailed below.
|
||||
|
||||
## Source
|
||||
|
||||
This plugin is part of [Awesome Copilot](https://github.com/github/awesome-copilot).
|
||||
|
||||
## Skills
|
||||
|
||||
### convert-word-to-md
|
||||
|
||||
Converts Word (`.docx`) documents to Markdown. Use it any time you want to read, summarize, review, compare, or extract information from a `.docx` file — even if you don't say "convert" explicitly.
|
||||
|
||||
> "Summarize this Word document."
|
||||
> "Extract all the action items from report.docx."
|
||||
> "Compare these two contracts."
|
||||
|
||||
### convert-excel-to-md
|
||||
|
||||
Converts Excel (`.xlsx`) workbooks to Markdown, rendering each sheet as a table. Use it any time you want to analyze, query, or summarize data in a spreadsheet — single file or a whole folder at once.
|
||||
|
||||
> "What are the top 5 rows by revenue in this spreadsheet?"
|
||||
> "Summarize all the worksheets in this workbook."
|
||||
> "Process every Excel file in this folder."
|
||||
|
||||
### convert-pdf-to-md
|
||||
|
||||
Converts PDF (`.pdf`) documents to Markdown, extracting both text and embedded images. Use it any time you want to read, summarize, or pull data from a PDF report, invoice, paper, or form.
|
||||
|
||||
> "Summarize this PDF."
|
||||
> "Extract all the dates mentioned in this contract."
|
||||
> "Process all the PDFs in this folder."
|
||||
|
||||
## License
|
||||
|
||||
MIT
|
||||
@@ -0,0 +1,131 @@
|
||||
---
|
||||
name: convert-excel-to-md
|
||||
description: 'Converts Excel (.xlsx) workbooks into Markdown so their contents can be accurately analyzed, summarized, searched, or extracted from. Use this skill whenever the user shares, references, or asks about a .xlsx file — even if they don''t say "convert" or "markdown" explicitly. This includes requests to "read", "summarize", "review", "extract data from", "compare", "chart", or "analyze" a spreadsheet, workbook, budget, data export, or tracker. Always run the bundled conversion script to produce Markdown first; do not attempt to parse .xlsx content directly or write ad-hoc extraction code. Also use this skill for batch requests involving a whole folder of Excel workbooks. IMPORTANT: When the user references a folder or set of documents containing multiple file types (.pdf, .docx, .xlsx), invoke ALL three sibling skills — convert-pdf-to-md, convert-word-to-md, and convert-excel-to-md — so no file type is silently skipped.'
|
||||
---
|
||||
|
||||
# Convert Excel to Markdown
|
||||
|
||||
## When to use this skill
|
||||
|
||||
Trigger this skill any time there is a `.xlsx` file that needs to be
|
||||
understood or processed — for example, a user attaches a spreadsheet and
|
||||
asks questions about it, wants a summary of the data, wants specific rows or
|
||||
values pulled out, or wants multiple workbooks in a folder processed
|
||||
together. Excel's native `.xlsx` format is a zipped XML bundle that is not
|
||||
reliably readable as plain text, so always convert it to Markdown first
|
||||
using the script in this skill rather than trying to open or parse the file
|
||||
directly.
|
||||
|
||||
This skill only supports `.xlsx`. If asked to convert a legacy `.xls` file,
|
||||
tell the user it isn't supported and ask them to re-save it as `.xlsx`
|
||||
(Excel: File > Save As > Excel Workbook (.xlsx)) first.
|
||||
|
||||
**Mixed file types:** When the user references a folder or set of documents
|
||||
containing multiple supported file types (`.pdf`, `.docx`, `.xlsx`), this
|
||||
skill handles only `.xlsx` files. The agent MUST also invoke the sibling
|
||||
skills in parallel:
|
||||
- `convert-pdf-to-md` for any `.pdf` files
|
||||
- `convert-word-to-md` for any `.docx` files
|
||||
|
||||
Never process a folder and silently skip a supported file type. All three
|
||||
skills must be invoked together when mixed types are present.
|
||||
|
||||
## Setup (once per environment)
|
||||
|
||||
Before the first conversion in a given environment, follow
|
||||
[`references/setup.md`](references/setup.md) step by step to ensure Python,
|
||||
pip, and the `markitdown` package are installed. Do this proactively rather
|
||||
than guessing whether the environment is ready — the script itself will
|
||||
also fail with a clear pointer back to that file if `markitdown` turns out
|
||||
to be missing, so it's safe to just try the conversion first if you're
|
||||
reasonably confident setup was already done.
|
||||
|
||||
## Usage
|
||||
|
||||
The conversion script lives at `scripts/convert_excel_to_md.py`.
|
||||
|
||||
**Output structure:** MarkItDown's XLSX converter renders each sheet as its
|
||||
own `## <SheetName>` Markdown table — it has no support for embedded images
|
||||
at all. This script separately extracts real embedded images (raster
|
||||
pictures, not charts) and maps them to the sheet they belong to, writing a
|
||||
self-contained folder per document:
|
||||
|
||||
```
|
||||
<name>/
|
||||
img/
|
||||
sheet001_<sheetname>_img001.<ext>
|
||||
sheet002_<sheetname>_img001.<ext>
|
||||
...
|
||||
<name>.md (each sheet's images appear right after its table,
|
||||
under a "#### Images in this sheet" heading)
|
||||
```
|
||||
|
||||
This is per-sheet placement, not exact cell position — the finest
|
||||
granularity MarkItDown's stable output anchors (the `## <SheetName>`
|
||||
headings) allow. If a workbook has no embedded images, no `img/` folder or
|
||||
image sections are created. Native Excel **charts** are not extracted as
|
||||
images (only actual embedded pictures are — charts would need to be
|
||||
rendered by Excel/LibreOffice, which this lightweight skill does not do).
|
||||
|
||||
**Single file:**
|
||||
|
||||
```powershell
|
||||
python scripts\convert_excel_to_md.py "C:\path\to\workbook.xlsx"
|
||||
```
|
||||
|
||||
This creates a `workbook\` folder next to the source file (containing
|
||||
`workbook.md` and, if present, `workbook\img\`). To control the destination
|
||||
folder explicitly:
|
||||
|
||||
```powershell
|
||||
python scripts\convert_excel_to_md.py "C:\path\to\workbook.xlsx" -o "C:\path\to\output_folder"
|
||||
```
|
||||
|
||||
**A folder of workbooks (batch mode):**
|
||||
|
||||
```powershell
|
||||
python scripts\convert_excel_to_md.py "C:\path\to\folder"
|
||||
```
|
||||
|
||||
Add `--recursive` to also include subfolders:
|
||||
|
||||
```powershell
|
||||
python scripts\convert_excel_to_md.py "C:\path\to\folder" --recursive
|
||||
```
|
||||
|
||||
Each `.xlsx` found gets its own `<name>\` output folder next to it by
|
||||
default. Pass `-o "C:\path\to\output_parent"` to collect all the generated
|
||||
`<name>\` folders under a separate parent directory instead (subfolder
|
||||
structure is preserved when combined with `--recursive`).
|
||||
|
||||
After conversion, read the resulting `.md` file(s) to perform the actual
|
||||
analysis the user asked for — the script's job is only to produce accurate
|
||||
Markdown (and images), not to interpret the content.
|
||||
|
||||
## Deciding where output goes
|
||||
|
||||
**Default — always output next to the source file.** The `<name>/` folder
|
||||
is created in the same directory as the source `.xlsx`. This is the required
|
||||
default for every case. Do NOT override it unless the user explicitly asks
|
||||
for a different location.
|
||||
|
||||
**Only use `-o` when** the user explicitly provides an output path (e.g.,
|
||||
"save the output to `C:\output`", "put the results in `D:\work`"). Do NOT
|
||||
pass `-o` based on the agent's current working directory, the session state
|
||||
folder, or any implied location.
|
||||
|
||||
**If the source file path cannot be fully resolved** — for example, the
|
||||
user provides only a filename with no directory, or the path is ambiguous —
|
||||
use `ask_user` to confirm the full absolute path before running the
|
||||
conversion. Never guess or assume the directory.
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
| Symptom | Likely cause | Fix |
|
||||
|---|---|---|
|
||||
| `ModuleNotFoundError: No module named 'markitdown'` / exit code 2 | MarkItDown not installed | Follow `references/setup.md` |
|
||||
| `ERROR: Unsupported file type '.xls'` / exit code 3 | Legacy `.xls`, not `.xlsx` | Ask the user to re-save as `.xlsx` |
|
||||
| `ERROR: Input path not found` / exit code 3 | Wrong path, or file moved | Confirm the correct path with the user |
|
||||
| `FAILED <file> -> ...` in batch output | That specific file is corrupt, password-protected, or otherwise unreadable | Report which file(s) failed; other files in the batch still succeed |
|
||||
| `NOTE: skipped N non-.xlsx file(s)` | Folder contains non-Excel files | Expected — those files are intentionally ignored |
|
||||
| A sheet's charts don't appear as images | Charts are chart objects, not embedded pictures — this skill only extracts real embedded raster images | Expected; mention this limitation if the user specifically needs chart images |
|
||||
@@ -0,0 +1,73 @@
|
||||
# Environment Setup for convert-excel-to-md
|
||||
|
||||
Follow these steps exactly, in order, before running `scripts/convert_excel_to_md.py`
|
||||
for the first time in a given environment. Don't skip steps or improvise
|
||||
alternatives — they're written to be deterministic and safe to re-run.
|
||||
|
||||
## 1. Check Python is available (3.10+)
|
||||
|
||||
```powershell
|
||||
python --version
|
||||
```
|
||||
|
||||
- If this fails (command not found), install Python 3.10 or newer:
|
||||
- Windows: `winget install --id Python.Python.3.12 -e`
|
||||
- macOS: `brew install python@3.12`
|
||||
- Linux (Debian/Ubuntu): `sudo apt-get update && sudo apt-get install -y python3 python3-pip python-is-python3`
|
||||
- If the reported version is older than 3.10, install a newer Python using
|
||||
the same command above (MarkItDown requires 3.10+).
|
||||
|
||||
## 2. Check pip is available
|
||||
|
||||
```powershell
|
||||
python -m pip --version
|
||||
```
|
||||
|
||||
- If this fails, bootstrap pip:
|
||||
|
||||
```powershell
|
||||
python -m ensurepip --upgrade
|
||||
```
|
||||
|
||||
## 3. Install MarkItDown with Excel (.xlsx) support
|
||||
|
||||
Use the `scripts/requirements.txt` file bundled with this skill to install a pinned,
|
||||
known-good version of the dependency:
|
||||
|
||||
```powershell
|
||||
python -m pip install -r scripts/requirements.txt
|
||||
```
|
||||
|
||||
This pulls in `markitdown[xlsx]` (MarkItDown's XLSX table conversion
|
||||
dependencies, which include `pandas` and `openpyxl`). No extra package is needed for image extraction — this
|
||||
skill's script reads embedded images directly from the `.xlsx` zip
|
||||
structure using Python's built-in `zipfile` and `xml` modules.
|
||||
|
||||
## 4. Verify the install
|
||||
|
||||
```powershell
|
||||
python -c "from markitdown import MarkItDown; print('markitdown OK')"
|
||||
```
|
||||
|
||||
Expect to see `markitdown OK` printed with no errors. If you see
|
||||
`ModuleNotFoundError: No module named 'markitdown'`, repeat step 3 — pip may
|
||||
be installing into a different Python environment than the one being
|
||||
invoked (check `python -m pip --version` shows the same path as `python
|
||||
--version`'s interpreter).
|
||||
|
||||
## Notes
|
||||
|
||||
- This setup only needs to be done once per environment/virtual environment,
|
||||
not once per conversion.
|
||||
- `convert_excel_to_md.py` itself also checks for `markitdown` at startup
|
||||
and prints a pointer back to this file if it's missing, so re-running
|
||||
setup is safe and idempotent.
|
||||
- Only `.xlsx` is supported by this skill. Legacy binary `.xls` files are
|
||||
out of scope (a completely different, harder-to-parse file format) — ask
|
||||
the user to re-save the file as `.xlsx` (Excel: File > Save As > Excel
|
||||
Workbook (.xlsx)) if one is encountered.
|
||||
- Chart objects (as opposed to embedded pictures) are not extracted as
|
||||
images — only raster pictures actually embedded in the workbook's
|
||||
`xl/media` folder are. Native Excel charts would need to be rendered by
|
||||
Excel/LibreOffice to become images, which this lightweight skill does not
|
||||
attempt.
|
||||
@@ -0,0 +1,354 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Convert Excel (.xlsx) workbooks to Markdown using Microsoft's MarkItDown,
|
||||
with embedded images extracted to real files and placed under the correct
|
||||
sheet (MarkItDown's XLSX converter only extracts sheet data as tables -- it
|
||||
has no support for embedded images at all).
|
||||
|
||||
Usage:
|
||||
python convert_excel_to_md.py <input> [-o OUTPUT] [--recursive]
|
||||
|
||||
<input> may be either:
|
||||
- a path to a single .xlsx file, or
|
||||
- a path to a directory (batch mode: every .xlsx file directly inside it
|
||||
is converted; pass --recursive to also descend into subdirectories).
|
||||
|
||||
Output:
|
||||
For each source .xlsx (named "<name>.xlsx"), a folder is created
|
||||
containing the Markdown and its images, in this layout:
|
||||
|
||||
<name>/
|
||||
img/
|
||||
Sheet1_img001.<ext>
|
||||
Sheet2_img001.<ext>
|
||||
...
|
||||
<name>.md
|
||||
|
||||
MarkItDown renders each sheet as its own "## <SheetName>" section with a
|
||||
Markdown table. This script independently maps embedded images to the
|
||||
sheet they belong to (via the .xlsx zip's drawing relationships) and
|
||||
inserts a "#### Images in this sheet" block right after that sheet's
|
||||
table, before the next "## " heading. This is per-sheet placement (not
|
||||
exact cell position), which is the finest granularity MarkItDown's stable
|
||||
output anchors allow.
|
||||
|
||||
- Single file mode: the "<name>/" folder is created next to the source
|
||||
file, or at -o/--output (treated as the exact destination folder) if
|
||||
given.
|
||||
- Batch/directory mode: a "<name>/" folder is created next to each source
|
||||
file, or under -o/--output (treated as a parent directory, created if
|
||||
missing) if given, preserving relative subfolder structure when
|
||||
--recursive is used.
|
||||
- If a workbook has no embedded images, no "img/" folder or "Images in
|
||||
this sheet" sections are created.
|
||||
|
||||
Exit codes:
|
||||
0 - all requested conversions succeeded
|
||||
1 - one or more conversions failed (partial success in batch mode)
|
||||
2 - required dependency ("markitdown") is not installed
|
||||
3 - invalid input (path not found, or single-file input is not .xlsx)
|
||||
"""
|
||||
import argparse
|
||||
import posixpath
|
||||
import re
|
||||
import shutil
|
||||
import sys
|
||||
import zipfile
|
||||
from pathlib import Path
|
||||
from xml.etree import ElementTree as ET
|
||||
|
||||
EXIT_OK = 0
|
||||
EXIT_CONVERSION_FAILED = 1
|
||||
EXIT_MISSING_DEPENDENCY = 2
|
||||
EXIT_INVALID_INPUT = 3
|
||||
|
||||
_REL_NS = "http://schemas.openxmlformats.org/package/2006/relationships"
|
||||
_MAIN_NS = "http://schemas.openxmlformats.org/spreadsheetml/2006/main"
|
||||
_R_NS = "http://schemas.openxmlformats.org/officeDocument/2006/relationships"
|
||||
_A_NS = "http://schemas.openxmlformats.org/drawingml/2006/main"
|
||||
|
||||
# Matches MarkItDown's per-sheet heading, e.g. "## Sheet1"
|
||||
_SHEET_HEADER_RE = re.compile(r"^## (.+)$", re.MULTILINE)
|
||||
|
||||
|
||||
def _import_markitdown():
|
||||
"""Import MarkItDown, failing with a clear, actionable message if absent."""
|
||||
try:
|
||||
from markitdown import MarkItDown
|
||||
return MarkItDown
|
||||
except ImportError:
|
||||
print(
|
||||
"ERROR: The 'markitdown' package is not installed.\n"
|
||||
"See references/setup.md for this skill, or run:\n"
|
||||
' pip install "markitdown[xlsx]"',
|
||||
file=sys.stderr,
|
||||
)
|
||||
sys.exit(EXIT_MISSING_DEPENDENCY)
|
||||
|
||||
|
||||
def _normalize_rel_path(base_dir: str, target: str) -> str:
|
||||
"""Resolve a (possibly relative, e.g. '../media/image1.png') relationship
|
||||
target against the directory containing the part that referenced it."""
|
||||
if target.startswith("/"):
|
||||
return target.lstrip("/")
|
||||
return posixpath.normpath(posixpath.join(base_dir, target))
|
||||
|
||||
|
||||
def _sheet_name_to_media(xlsx_path: Path):
|
||||
"""Return {sheet_name: [media_zip_path, ...]} in per-sheet document
|
||||
order, by walking workbook.xml -> worksheet -> drawing -> media
|
||||
relationships. Returns {} if anything is missing/malformed (falls back
|
||||
gracefully -- images just won't be extracted for that sheet)."""
|
||||
try:
|
||||
with zipfile.ZipFile(xlsx_path) as z:
|
||||
names = set(z.namelist())
|
||||
if "xl/workbook.xml" not in names or "xl/_rels/workbook.xml.rels" not in names:
|
||||
return {}
|
||||
workbook_xml = z.read("xl/workbook.xml")
|
||||
workbook_rels_xml = z.read("xl/_rels/workbook.xml.rels")
|
||||
|
||||
sheet_rid = {}
|
||||
for sheet_el in ET.fromstring(workbook_xml).iter(f"{{{_MAIN_NS}}}sheet"):
|
||||
name = sheet_el.get("name")
|
||||
rid = sheet_el.get(f"{{{_R_NS}}}id")
|
||||
if name and rid:
|
||||
sheet_rid[name] = rid
|
||||
|
||||
rid_target = {}
|
||||
for rel in ET.fromstring(workbook_rels_xml).findall(f"{{{_REL_NS}}}Relationship"):
|
||||
rid_target[rel.get("Id")] = rel.get("Target")
|
||||
|
||||
result = {}
|
||||
for sheet_name, rid in sheet_rid.items():
|
||||
target = rid_target.get(rid)
|
||||
if not target:
|
||||
continue
|
||||
# workbook.xml.rels targets are typically relative to "xl/",
|
||||
# but OOXML allows package-absolute targets (leading "/") too.
|
||||
sheet_path = _normalize_rel_path("xl", target)
|
||||
if sheet_path not in names or "/" not in sheet_path:
|
||||
continue
|
||||
sheet_dir, sheet_file = sheet_path.rsplit("/", 1)
|
||||
sheet_rels_path = f"{sheet_dir}/_rels/{sheet_file}.rels"
|
||||
if sheet_rels_path not in names:
|
||||
continue
|
||||
|
||||
drawing_rid = None
|
||||
for d in ET.fromstring(z.read(sheet_path)).iter(f"{{{_MAIN_NS}}}drawing"):
|
||||
drawing_rid = d.get(f"{{{_R_NS}}}id")
|
||||
break
|
||||
if not drawing_rid:
|
||||
continue
|
||||
|
||||
drawing_target = None
|
||||
for rel in ET.fromstring(z.read(sheet_rels_path)).findall(f"{{{_REL_NS}}}Relationship"):
|
||||
if rel.get("Id") == drawing_rid:
|
||||
drawing_target = rel.get("Target")
|
||||
break
|
||||
if not drawing_target:
|
||||
continue
|
||||
drawing_path = _normalize_rel_path(sheet_dir, drawing_target)
|
||||
if drawing_path not in names or "/" not in drawing_path:
|
||||
continue
|
||||
drawing_dir, drawing_file = drawing_path.rsplit("/", 1)
|
||||
drawing_rels_path = f"{drawing_dir}/_rels/{drawing_file}.rels"
|
||||
if drawing_rels_path not in names:
|
||||
continue
|
||||
|
||||
drawing_rel_map = {}
|
||||
for rel in ET.fromstring(z.read(drawing_rels_path)).findall(f"{{{_REL_NS}}}Relationship"):
|
||||
drawing_rel_map[rel.get("Id")] = rel.get("Target")
|
||||
|
||||
media_paths = []
|
||||
for blip in ET.fromstring(z.read(drawing_path)).iter(f"{{{_A_NS}}}blip"):
|
||||
embed_rid = blip.get(f"{{{_R_NS}}}embed")
|
||||
if not embed_rid:
|
||||
continue
|
||||
rel_target = drawing_rel_map.get(embed_rid)
|
||||
if not rel_target:
|
||||
continue
|
||||
media_path = _normalize_rel_path(drawing_dir, rel_target)
|
||||
if media_path in names:
|
||||
media_paths.append(media_path)
|
||||
|
||||
if media_paths:
|
||||
result[sheet_name] = media_paths
|
||||
return result
|
||||
except (zipfile.BadZipFile, KeyError, OSError, ET.ParseError):
|
||||
return {}
|
||||
|
||||
|
||||
def _sanitize_filename_part(name: str) -> str:
|
||||
safe = re.sub(r"[^A-Za-z0-9_.-]+", "_", name).strip("_")
|
||||
return safe or "sheet"
|
||||
|
||||
|
||||
def extract_images(xlsx_path: Path, img_dir: Path):
|
||||
"""Extract embedded images from xlsx_path, grouped by sheet name.
|
||||
Returns {sheet_name: [filename, ...]} in per-sheet order. Files are
|
||||
named '<sanitized_sheet_name>_img{N:03d}.<ext>'."""
|
||||
sheet_media = _sheet_name_to_media(xlsx_path)
|
||||
if not sheet_media:
|
||||
return {}
|
||||
|
||||
written = {}
|
||||
with zipfile.ZipFile(xlsx_path) as z:
|
||||
names_in_zip = set(z.namelist())
|
||||
for sheet_idx, (sheet_name, media_paths) in enumerate(sheet_media.items(), start=1):
|
||||
safe_name = f"sheet{sheet_idx:03d}_{_sanitize_filename_part(sheet_name)}"
|
||||
files = []
|
||||
for idx, media_path in enumerate(media_paths, start=1):
|
||||
if media_path not in names_in_zip:
|
||||
print(f"WARNING: {media_path} not found in {xlsx_path}", file=sys.stderr)
|
||||
continue
|
||||
ext = Path(media_path).suffix.lstrip(".").lower() or "bin"
|
||||
if ext == "jpg":
|
||||
ext = "jpeg"
|
||||
out_name = f"{safe_name}_img{idx:03d}.{ext}"
|
||||
img_dir.mkdir(parents=True, exist_ok=True)
|
||||
(img_dir / out_name).write_bytes(z.read(media_path))
|
||||
files.append(out_name)
|
||||
if files:
|
||||
written[sheet_name] = files
|
||||
return written
|
||||
|
||||
|
||||
def insert_sheet_images(markdown_text: str, sheet_images) -> str:
|
||||
"""Insert a '#### Images in this sheet' block right after each sheet's
|
||||
section (before the next '## ' heading or end of text). If a sheet has
|
||||
no images, or no '## ' headings are found at all, the text is returned
|
||||
unchanged for that part."""
|
||||
if not sheet_images:
|
||||
return markdown_text
|
||||
matches = list(_SHEET_HEADER_RE.finditer(markdown_text))
|
||||
if not matches:
|
||||
return markdown_text
|
||||
|
||||
pieces = []
|
||||
last_end = 0
|
||||
for i, m in enumerate(matches):
|
||||
sheet_name = m.group(1).removesuffix("\r")
|
||||
start = m.start()
|
||||
end = matches[i + 1].start() if i + 1 < len(matches) else len(markdown_text)
|
||||
pieces.append(markdown_text[last_end:start])
|
||||
section = markdown_text[start:end].rstrip("\n")
|
||||
images = sheet_images.get(sheet_name)
|
||||
if images:
|
||||
section += "\n\n#### Images in this sheet\n\n"
|
||||
section += "\n".join(f"" for name in images)
|
||||
pieces.append(section + "\n\n")
|
||||
last_end = end
|
||||
pieces.append(markdown_text[last_end:])
|
||||
return "".join(pieces).rstrip() + "\n"
|
||||
|
||||
|
||||
def convert_one(md, source: Path, dest_dir: Path) -> bool:
|
||||
"""Convert a single .xlsx file to a '<name>/' folder containing the
|
||||
Markdown file and an 'img/' folder of extracted images. Returns True on
|
||||
success."""
|
||||
try:
|
||||
result = md.convert(str(source))
|
||||
except Exception as exc: # noqa: BLE001 - surface any conversion error
|
||||
print(f"FAILED {source} -> {exc}", file=sys.stderr)
|
||||
return False
|
||||
|
||||
try:
|
||||
img_dir = dest_dir / "img"
|
||||
if dest_dir.exists():
|
||||
if img_dir.exists():
|
||||
shutil.rmtree(img_dir)
|
||||
dest_dir.mkdir(parents=True, exist_ok=True)
|
||||
sheet_images = extract_images(source, img_dir)
|
||||
text = insert_sheet_images(result.text_content, sheet_images)
|
||||
md_path = dest_dir / f"{source.stem}.md"
|
||||
md_path.write_text(text, encoding="utf-8")
|
||||
except OSError as exc:
|
||||
print(f"FAILED {source} -> could not write output in {dest_dir}: {exc}", file=sys.stderr)
|
||||
return False
|
||||
|
||||
img_count = sum(len(v) for v in sheet_images.values())
|
||||
img_note = f", {img_count} image(s)" if img_count else ""
|
||||
print(f"OK {source} -> {md_path}{img_note}")
|
||||
return True
|
||||
|
||||
|
||||
def find_xlsx_files(root: Path, recursive: bool):
|
||||
"""Return (xlsx_files, skipped_count) for files directly/recursively under root."""
|
||||
pattern_iter = root.rglob("*") if recursive else root.iterdir()
|
||||
xlsx_files = []
|
||||
skipped = 0
|
||||
for entry in pattern_iter:
|
||||
if entry.is_dir():
|
||||
continue
|
||||
if entry.suffix.lower() == ".xlsx":
|
||||
xlsx_files.append(entry)
|
||||
else:
|
||||
skipped += 1
|
||||
return sorted(xlsx_files), skipped
|
||||
|
||||
|
||||
def main() -> int:
|
||||
parser = argparse.ArgumentParser(description=__doc__.splitlines()[0])
|
||||
parser.add_argument("input", help="Path to a .xlsx file or a directory of .xlsx files")
|
||||
parser.add_argument(
|
||||
"-o", "--output",
|
||||
help=(
|
||||
"Destination folder for the '<name>/' output (single-file mode), "
|
||||
"or parent directory under which each '<name>/' output folder is "
|
||||
"created (batch mode)"
|
||||
),
|
||||
)
|
||||
parser.add_argument(
|
||||
"--recursive", action="store_true",
|
||||
help="When input is a directory, also search subdirectories",
|
||||
)
|
||||
args = parser.parse_args()
|
||||
|
||||
#MarkItDown = _import_markitdown()
|
||||
#md = MarkItDown()
|
||||
|
||||
source = Path(args.input)
|
||||
if not source.exists():
|
||||
print(f"ERROR: Input path not found: {source}", file=sys.stderr)
|
||||
return EXIT_INVALID_INPUT
|
||||
|
||||
if source.is_file() and source.suffix.lower() != ".xlsx":
|
||||
print(
|
||||
f"ERROR: Unsupported file type '{source.suffix}'. "
|
||||
"This skill only converts .xlsx files.",
|
||||
file=sys.stderr,
|
||||
)
|
||||
return EXIT_INVALID_INPUT
|
||||
|
||||
MarkItDown = _import_markitdown()
|
||||
md = MarkItDown()
|
||||
|
||||
if source.is_file():
|
||||
dest_dir = Path(args.output) if args.output else source.parent / source.stem
|
||||
return EXIT_OK if convert_one(md, source, dest_dir) else EXIT_CONVERSION_FAILED
|
||||
|
||||
# Directory / batch mode
|
||||
xlsx_files, skipped = find_xlsx_files(source, args.recursive)
|
||||
if skipped:
|
||||
print(f"NOTE: skipped {skipped} non-.xlsx file(s) in {source}")
|
||||
if not xlsx_files:
|
||||
print(f"ERROR: No .xlsx files found under {source}", file=sys.stderr)
|
||||
return EXIT_INVALID_INPUT
|
||||
|
||||
out_dir = Path(args.output) if args.output else None
|
||||
success_count = 0
|
||||
for xlsx_path in xlsx_files:
|
||||
if out_dir is not None:
|
||||
rel = xlsx_path.relative_to(source)
|
||||
dest_dir = out_dir / rel.parent / xlsx_path.stem
|
||||
else:
|
||||
dest_dir = xlsx_path.parent / xlsx_path.stem
|
||||
if convert_one(md, xlsx_path, dest_dir):
|
||||
success_count += 1
|
||||
|
||||
total = len(xlsx_files)
|
||||
print(f"\nConverted {success_count}/{total} file(s).")
|
||||
return EXIT_OK if success_count == total else EXIT_CONVERSION_FAILED
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
@@ -0,0 +1 @@
|
||||
markitdown[xlsx]>=0.1.0
|
||||
@@ -0,0 +1,130 @@
|
||||
---
|
||||
name: convert-pdf-to-md
|
||||
description: 'Converts PDF (.pdf) documents into Markdown so their contents can be accurately analyzed, summarized, searched, or extracted from. Use this skill whenever the user shares, references, or asks about a .pdf file — even if they don''t say "convert" or "markdown" explicitly. This includes requests to "read", "summarize", "review", "extract data from", "compare", or "analyze" a PDF report, paper, invoice, form, contract, or scanned document. Always run the bundled conversion script to produce Markdown first; do not attempt to parse PDF content directly or write ad-hoc extraction code. Also use this skill for batch requests involving a whole folder of PDF documents. IMPORTANT: When the user references a folder or set of documents containing multiple file types (.pdf, .docx, .xlsx), invoke ALL three sibling skills — convert-pdf-to-md, convert-word-to-md, and convert-excel-to-md — so no file type is silently skipped.'
|
||||
---
|
||||
|
||||
# Convert PDF to Markdown
|
||||
|
||||
## When to use this skill
|
||||
|
||||
Trigger this skill any time there is a `.pdf` file that needs to be
|
||||
understood or processed — for example, a user attaches a PDF and asks
|
||||
questions about it, wants a summary, wants specific data or tables pulled
|
||||
out, or wants multiple PDFs in a folder processed together. PDF is a
|
||||
layout/print format, not reliably readable as plain text, so always convert
|
||||
it to Markdown first using the script in this skill rather than trying to
|
||||
open or parse the file directly.
|
||||
|
||||
This skill only supports `.pdf` — that's MarkItDown's only PDF-family
|
||||
format, so there's no legacy format to worry about here (unlike Word's
|
||||
`.doc` or Excel's `.xls`).
|
||||
|
||||
**Mixed file types:** When the user references a folder or set of documents
|
||||
containing multiple supported file types (`.pdf`, `.docx`, `.xlsx`), this
|
||||
skill handles only `.pdf` files. The agent MUST also invoke the sibling
|
||||
skills in parallel:
|
||||
- `convert-word-to-md` for any `.docx` files
|
||||
- `convert-excel-to-md` for any `.xlsx` files
|
||||
|
||||
Never process a folder and silently skip a supported file type. All three
|
||||
skills must be invoked together when mixed types are present.
|
||||
|
||||
## Setup (once per environment)
|
||||
|
||||
Before the first conversion in a given environment, follow
|
||||
[`references/setup.md`](references/setup.md) step by step to ensure Python,
|
||||
pip, `markitdown`, and `pymupdf` (for image extraction) are installed. Do
|
||||
this proactively rather than guessing whether the environment is ready — the
|
||||
script itself will also fail with a clear pointer back to that file if a
|
||||
dependency turns out to be missing, so it's safe to just try the conversion
|
||||
first if you're reasonably confident setup was already done.
|
||||
|
||||
## Usage
|
||||
|
||||
The conversion script lives at `scripts/convert_pdf_to_md.py`.
|
||||
|
||||
**Output structure:** MarkItDown's PDF converter extracts text and tables
|
||||
only — it has no concept of embedded images at all. This script separately
|
||||
extracts real embedded images via PyMuPDF and writes a self-contained folder
|
||||
per document:
|
||||
|
||||
```
|
||||
<name>/
|
||||
img/
|
||||
page001_img001.<ext>
|
||||
page002_img001.<ext>
|
||||
...
|
||||
<name>.md
|
||||
```
|
||||
|
||||
Because MarkItDown's PDF text does not preserve reliable per-page markers,
|
||||
there's no safe way to know exactly where inline an image belongs. Rather
|
||||
than risk misplacing images next to the wrong paragraph, the script appends
|
||||
a `## Extracted Images` section at the end of the Markdown, with a
|
||||
`### Page N` subheading per page that has images — read this section
|
||||
separately from the main body text. If the document has no embedded images,
|
||||
no `img/` folder or `Extracted Images` section is created.
|
||||
|
||||
**Single file:**
|
||||
|
||||
```powershell
|
||||
python scripts\convert_pdf_to_md.py "C:\path\to\document.pdf"
|
||||
```
|
||||
|
||||
This creates a `document\` folder next to the source file (containing
|
||||
`document.md` and, if present, `document\img\`). To control the destination
|
||||
folder explicitly:
|
||||
|
||||
```powershell
|
||||
python scripts\convert_pdf_to_md.py "C:\path\to\document.pdf" -o "C:\path\to\output_folder"
|
||||
```
|
||||
|
||||
**A folder of PDFs (batch mode):**
|
||||
|
||||
```powershell
|
||||
python scripts\convert_pdf_to_md.py "C:\path\to\folder"
|
||||
```
|
||||
|
||||
Add `--recursive` to also include subfolders:
|
||||
|
||||
```powershell
|
||||
python scripts\convert_pdf_to_md.py "C:\path\to\folder" --recursive
|
||||
```
|
||||
|
||||
Each `.pdf` found gets its own `<name>\` output folder next to it by
|
||||
default. Pass `-o "C:\path\to\output_parent"` to collect all the generated
|
||||
`<name>\` folders under a separate parent directory instead (subfolder
|
||||
structure is preserved when combined with `--recursive`).
|
||||
|
||||
After conversion, read the resulting `.md` file(s) to perform the actual
|
||||
analysis the user asked for — the script's job is only to produce accurate
|
||||
Markdown (and images), not to interpret the content.
|
||||
|
||||
## Deciding where output goes
|
||||
|
||||
**Default — always output next to the source file.** The `<name>/` folder
|
||||
is created in the same directory as the source `.pdf`. This is the required
|
||||
default for every case. Do NOT override it unless the user explicitly asks
|
||||
for a different location.
|
||||
|
||||
**Only use `-o` when** the user explicitly provides an output path (e.g.,
|
||||
"save the output to `C:\output`", "put the results in `D:\work`"). Do NOT
|
||||
pass `-o` based on the agent's current working directory, the session state
|
||||
folder, or any implied location.
|
||||
|
||||
**If the source file path cannot be fully resolved** — for example, the
|
||||
user provides only a filename with no directory, or the path is ambiguous —
|
||||
use `ask_user` to confirm the full absolute path before running the
|
||||
conversion. Never guess or assume the directory.
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
| Symptom | Likely cause | Fix |
|
||||
|---|---|---|
|
||||
| `ModuleNotFoundError: No module named 'markitdown'` or `'fitz'` / exit code 2 | MarkItDown or PyMuPDF not installed | Follow `references/setup.md` |
|
||||
| `ERROR: Unsupported file type '...'` / exit code 3 | Not a `.pdf` file | Ask the user for the correct file, or if it's `.doc`/`.docx`/`.xlsx`, use the matching sibling skill instead |
|
||||
| `ERROR: Input path not found` / exit code 3 | Wrong path, or file moved | Confirm the correct path with the user |
|
||||
| `FAILED <file> -> ...` in batch output | That specific file is corrupt, password-protected, or otherwise unreadable | Report which file(s) failed; other files in the batch still succeed |
|
||||
| `NOTE: skipped N non-.pdf file(s)` | Folder contains non-PDF files | Expected — those files are intentionally ignored |
|
||||
| Markdown body is empty or near-empty despite images being extracted | The PDF is scanned/image-only with no embedded text layer; MarkItDown does not perform OCR | Tell the user OCR isn't supported — the extracted page images are still available for them to view |
|
||||
| Images appear in an appendix instead of inline with the text | Deliberate limitation — MarkItDown's PDF text has no reliable per-page markers to place images inline | Expected behavior; cross-reference the `### Page N` heading with the surrounding text context if needed |
|
||||
@@ -0,0 +1,71 @@
|
||||
# Environment Setup for convert-pdf-to-md
|
||||
|
||||
Follow these steps exactly, in order, before running `scripts/convert_pdf_to_md.py`
|
||||
for the first time in a given environment. Don't skip steps or improvise
|
||||
alternatives — they're written to be deterministic and safe to re-run.
|
||||
|
||||
## 1. Check Python is available (3.10+)
|
||||
|
||||
```powershell
|
||||
python --version
|
||||
```
|
||||
|
||||
- If this fails (command not found), install Python 3.10 or newer:
|
||||
- Windows: `winget install --id Python.Python.3.12 -e`
|
||||
- macOS: `brew install python@3.12`
|
||||
- Linux (Debian/Ubuntu): `sudo apt-get update && sudo apt-get install -y python3 python3-pip python-is-python3`
|
||||
- If the reported version is older than 3.10, install a newer Python using
|
||||
the same command above (MarkItDown requires 3.10+).
|
||||
|
||||
## 2. Check pip is available
|
||||
|
||||
```powershell
|
||||
python -m pip --version
|
||||
```
|
||||
|
||||
- If this fails, bootstrap pip:
|
||||
|
||||
```powershell
|
||||
python -m ensurepip --upgrade
|
||||
```
|
||||
|
||||
## 3. Install MarkItDown with PDF support, plus PyMuPDF for image extraction
|
||||
|
||||
Use the `scripts/requirements.txt` file bundled with this skill to install pinned,
|
||||
known-good versions of the dependencies:
|
||||
|
||||
```powershell
|
||||
python -m pip install -r scripts/requirements.txt
|
||||
```
|
||||
|
||||
This pulls in `markitdown[pdf]` and `pymupdf>=1.24.0`. PyMuPDF (imported as `fitz`)
|
||||
is required separately because MarkItDown's PDF
|
||||
converter only extracts text and tables — it has no support for embedded
|
||||
images at all, so this skill's script extracts them itself.
|
||||
|
||||
## 4. Verify the install
|
||||
|
||||
```powershell
|
||||
python -c "from markitdown import MarkItDown; import fitz; print('markitdown + pymupdf OK')"
|
||||
```
|
||||
|
||||
Expect to see `markitdown + pymupdf OK` printed with no errors. If you see a
|
||||
`ModuleNotFoundError`, repeat step 3 — pip may be installing into a
|
||||
different Python environment than the one being invoked (check
|
||||
`python -m pip --version` shows the same path as `python --version`'s
|
||||
interpreter).
|
||||
|
||||
## Notes
|
||||
|
||||
- This setup only needs to be done once per environment/virtual environment,
|
||||
not once per conversion.
|
||||
- `convert_pdf_to_md.py` itself also checks for `markitdown` and `fitz` at
|
||||
startup and prints a pointer back to this file if either is missing, so
|
||||
re-running setup is safe and idempotent.
|
||||
- Only `.pdf` is supported by this skill — it's MarkItDown's only PDF-family
|
||||
format, so there's no legacy-format equivalent to worry about (unlike
|
||||
Word's `.doc` or Excel's `.xls`).
|
||||
- Scanned/image-only PDFs (no embedded text layer) will produce little or
|
||||
no text from MarkItDown, since it does not perform OCR. The images
|
||||
themselves will still be extracted and appended, but the text body may be
|
||||
empty or near-empty in that case — mention this to the user if it happens.
|
||||
@@ -0,0 +1,321 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Convert PDF documents to Markdown using Microsoft's MarkItDown, with
|
||||
embedded images extracted to real files via PyMuPDF (MarkItDown's PDF
|
||||
converter only extracts text/tables -- it does not detect or emit anything
|
||||
for embedded images at all).
|
||||
|
||||
Usage:
|
||||
python convert_pdf_to_md.py <input> [-o OUTPUT] [--recursive]
|
||||
|
||||
<input> may be either:
|
||||
- a path to a single .pdf file, or
|
||||
- a path to a directory (batch mode: every .pdf file directly inside it
|
||||
is converted; pass --recursive to also descend into subdirectories).
|
||||
|
||||
Output:
|
||||
For each source .pdf (named "<name>.pdf"), a folder is created containing
|
||||
the Markdown and its images, in this layout:
|
||||
|
||||
<name>/
|
||||
img/
|
||||
page001_img001.<ext>
|
||||
page001_img002.<ext>
|
||||
page002_img001.<ext>
|
||||
...
|
||||
<name>.md
|
||||
|
||||
IMPORTANT: MarkItDown's PDF text extraction does not preserve reliable
|
||||
per-page markers in the returned Markdown (pages are simply joined
|
||||
together, or in some cases returned as a single unmarked block of text).
|
||||
That means there is no safe way to know exactly where, inline, an image
|
||||
should go. Rather than guess and risk misplacing an image next to the
|
||||
wrong paragraph, this script appends a clearly labeled "## Extracted
|
||||
Images" section at the end of the Markdown, with a "### Page N"
|
||||
subheading per page that contains images. This is a deliberate, honest
|
||||
tradeoff -- read the images section separately from the main body text.
|
||||
|
||||
- Single file mode: the "<name>/" folder is created next to the source
|
||||
file, or at -o/--output (treated as the exact destination folder) if
|
||||
given.
|
||||
- Batch/directory mode: a "<name>/" folder is created next to each source
|
||||
file, or under -o/--output (treated as a parent directory, created if
|
||||
missing) if given, preserving relative subfolder structure when
|
||||
--recursive is used.
|
||||
- If a document has no embedded images, no "img/" folder or "Extracted
|
||||
Images" section is created.
|
||||
|
||||
Exit codes:
|
||||
0 - all requested conversions succeeded
|
||||
1 - one or more conversions failed (partial success in batch mode)
|
||||
2 - a required dependency ("markitdown" or "pymupdf") is not installed
|
||||
3 - invalid input (path not found, or single-file input is not .pdf)
|
||||
"""
|
||||
import argparse
|
||||
import sys
|
||||
import hashlib
|
||||
import shutil
|
||||
from pathlib import Path
|
||||
|
||||
EXIT_OK = 0
|
||||
EXIT_CONVERSION_FAILED = 1
|
||||
EXIT_MISSING_DEPENDENCY = 2
|
||||
EXIT_INVALID_INPUT = 3
|
||||
|
||||
|
||||
def _import_markitdown():
|
||||
"""Import MarkItDown, failing with a clear, actionable message if absent."""
|
||||
try:
|
||||
from markitdown import MarkItDown
|
||||
return MarkItDown
|
||||
except ImportError:
|
||||
print(
|
||||
"ERROR: The 'markitdown' package is not installed.\n"
|
||||
"See references/setup.md for this skill, or run:\n"
|
||||
' pip install "markitdown[pdf]"',
|
||||
file=sys.stderr,
|
||||
)
|
||||
sys.exit(EXIT_MISSING_DEPENDENCY)
|
||||
|
||||
|
||||
def _import_fitz():
|
||||
"""Import PyMuPDF (module name 'fitz'), failing with a clear message if absent."""
|
||||
try:
|
||||
import fitz
|
||||
import hashlib
|
||||
return fitz
|
||||
except ImportError:
|
||||
print(
|
||||
"ERROR: The 'pymupdf' package is not installed (needed for image "
|
||||
"extraction).\nSee references/setup.md for this skill, or run:\n"
|
||||
" pip install pymupdf",
|
||||
file=sys.stderr,
|
||||
)
|
||||
sys.exit(EXIT_MISSING_DEPENDENCY)
|
||||
|
||||
|
||||
def extract_images(fitz, pdf_path: Path, img_dir: Path):
|
||||
"""Extract embedded images from pdf_path, grouped by 1-based page number.
|
||||
Returns {page_num: [filename, ...]} in per-page image order. Files are
|
||||
named 'page{P:03d}_img{N:03d}.<ext>'. Corrupt/unreadable images are
|
||||
skipped with a warning rather than aborting the whole conversion.
|
||||
|
||||
Two sources are combined and deduplicated:
|
||||
1. Image XObjects via page.get_images(full=True) -- covers most embedded
|
||||
images in modern PDFs.
|
||||
2. Inline image blocks via page.get_text("dict") -- covers images stored
|
||||
directly in the page content stream, which get_images() misses entirely.
|
||||
Deduplication is by image bytes hash so the same raster is never written twice
|
||||
on the same page regardless of which source reported it."""
|
||||
written_by_page = {}
|
||||
try:
|
||||
doc = fitz.open(str(pdf_path))
|
||||
except Exception as exc: # noqa: BLE001
|
||||
print(f"WARNING: could not open {pdf_path} for image extraction: {exc}", file=sys.stderr)
|
||||
return written_by_page
|
||||
|
||||
try:
|
||||
for page_index in range(len(doc)):
|
||||
page = doc[page_index]
|
||||
page_label = page_index + 1
|
||||
seen_hashes: set = set()
|
||||
raw_images: list[tuple[bytes, str]] = [] # (image_bytes, ext)
|
||||
|
||||
# --- Source 1: XObject images ---
|
||||
try:
|
||||
xobjects = page.get_images(full=True)
|
||||
except Exception as exc: # noqa: BLE001
|
||||
print(
|
||||
f"WARNING: failed to enumerate XObject images on page {page_label} "
|
||||
f"of {pdf_path}: {exc}",
|
||||
file=sys.stderr,
|
||||
)
|
||||
xobjects = []
|
||||
|
||||
for img in xobjects:
|
||||
xref = img[0]
|
||||
try:
|
||||
base_image = doc.extract_image(xref)
|
||||
except Exception as exc: # noqa: BLE001
|
||||
print(
|
||||
f"WARNING: failed to extract XObject image xref={xref} on page "
|
||||
f"{page_label} of {pdf_path}: {exc}",
|
||||
file=sys.stderr,
|
||||
)
|
||||
continue
|
||||
img_bytes = base_image.get("image") or b""
|
||||
if not img_bytes:
|
||||
continue
|
||||
ext = (base_image.get("ext") or "png").lower()
|
||||
raw_images.append((img_bytes, ext))
|
||||
|
||||
# --- Source 2: Inline images via get_text("dict") ---
|
||||
try:
|
||||
blocks = page.get_text("dict", flags=fitz.TEXT_PRESERVE_IMAGES).get("blocks", [])
|
||||
except Exception as exc: # noqa: BLE001
|
||||
print(
|
||||
f"WARNING: failed to extract text/image dict on page {page_label} "
|
||||
f"of {pdf_path}: {exc}",
|
||||
file=sys.stderr,
|
||||
)
|
||||
blocks = []
|
||||
|
||||
for block in blocks:
|
||||
# Image blocks have type == 1
|
||||
if block.get("type") != 1:
|
||||
continue
|
||||
img_bytes = block.get("image") or b""
|
||||
if not img_bytes:
|
||||
continue
|
||||
# Derive extension from the block's "ext" key (fitz sets this)
|
||||
ext = (block.get("ext") or "png").lower()
|
||||
raw_images.append((img_bytes, ext))
|
||||
|
||||
# --- Write deduplicated images ---
|
||||
page_files = []
|
||||
img_idx = 1
|
||||
for img_bytes, ext in raw_images:
|
||||
h = hashlib.sha256(img_bytes).digest()
|
||||
if h in seen_hashes:
|
||||
continue
|
||||
seen_hashes.add(h)
|
||||
out_name = f"page{page_label:03d}_img{img_idx:03d}.{ext}"
|
||||
img_dir.mkdir(parents=True, exist_ok=True)
|
||||
(img_dir / out_name).write_bytes(img_bytes)
|
||||
page_files.append(out_name)
|
||||
img_idx += 1
|
||||
|
||||
if page_files:
|
||||
written_by_page[page_label] = page_files
|
||||
finally:
|
||||
doc.close()
|
||||
|
||||
return written_by_page
|
||||
|
||||
|
||||
def build_image_appendix(written_by_page) -> str:
|
||||
"""Build the '## Extracted Images' appendix text. Returns "" if empty."""
|
||||
if not written_by_page:
|
||||
return ""
|
||||
lines = ["", "## Extracted Images", ""]
|
||||
for page_num in sorted(written_by_page):
|
||||
lines.append(f"### Page {page_num}")
|
||||
lines.append("")
|
||||
for name in written_by_page[page_num]:
|
||||
lines.append(f"")
|
||||
lines.append("")
|
||||
return "\n".join(lines).rstrip() + "\n"
|
||||
|
||||
|
||||
def convert_one(md, fitz, source: Path, dest_dir: Path) -> bool:
|
||||
"""Convert a single .pdf file to a '<name>/' folder containing the
|
||||
Markdown file and an 'img/' folder of extracted images. Returns True on
|
||||
success."""
|
||||
try:
|
||||
result = md.convert(str(source))
|
||||
except Exception as exc: # noqa: BLE001 - surface any conversion error
|
||||
print(f"FAILED {source} -> {exc}", file=sys.stderr)
|
||||
return False
|
||||
|
||||
try:
|
||||
if dest_dir.exists():
|
||||
shutil.rmtree(dest_dir)
|
||||
dest_dir.mkdir(parents=True, exist_ok=True)
|
||||
written_by_page = extract_images(fitz, source, dest_dir / "img")
|
||||
appendix = build_image_appendix(written_by_page)
|
||||
text = result.text_content.rstrip("\n")
|
||||
full_text = f"{text}\n{appendix}" if appendix else f"{text}\n"
|
||||
md_path = dest_dir / f"{source.stem}.md"
|
||||
md_path.write_text(full_text, encoding="utf-8")
|
||||
except OSError as exc:
|
||||
print(f"FAILED {source} -> could not write output in {dest_dir}: {exc}", file=sys.stderr)
|
||||
return False
|
||||
|
||||
img_count = sum(len(v) for v in written_by_page.values())
|
||||
img_note = f", {img_count} image(s)" if img_count else ""
|
||||
print(f"OK {source} -> {md_path}{img_note}")
|
||||
return True
|
||||
|
||||
|
||||
def find_pdf_files(root: Path, recursive: bool):
|
||||
"""Return (pdf_files, skipped_count) for files directly/recursively under root."""
|
||||
pattern_iter = root.rglob("*") if recursive else root.iterdir()
|
||||
pdf_files = []
|
||||
skipped = 0
|
||||
for entry in pattern_iter:
|
||||
if entry.is_dir():
|
||||
continue
|
||||
if entry.suffix.lower() == ".pdf":
|
||||
pdf_files.append(entry)
|
||||
else:
|
||||
skipped += 1
|
||||
return sorted(pdf_files), skipped
|
||||
|
||||
|
||||
def main() -> int:
|
||||
parser = argparse.ArgumentParser(description=__doc__.splitlines()[0])
|
||||
parser.add_argument("input", help="Path to a .pdf file or a directory of .pdf files")
|
||||
parser.add_argument(
|
||||
"-o", "--output",
|
||||
help=(
|
||||
"Destination folder for the '<name>/' output (single-file mode), "
|
||||
"or parent directory under which each '<name>/' output folder is "
|
||||
"created (batch mode)"
|
||||
),
|
||||
)
|
||||
parser.add_argument(
|
||||
"--recursive", action="store_true",
|
||||
help="When input is a directory, also search subdirectories",
|
||||
)
|
||||
args = parser.parse_args()
|
||||
|
||||
#MarkItDown = _import_markitdown()
|
||||
#fitz = _import_fitz()
|
||||
#md = MarkItDown()
|
||||
|
||||
source = Path(args.input)
|
||||
if not source.exists():
|
||||
print(f"ERROR: Input path not found: {source}", file=sys.stderr)
|
||||
return EXIT_INVALID_INPUT
|
||||
|
||||
if source.is_file() and source.suffix.lower() != ".pdf":
|
||||
print(
|
||||
f"ERROR: Unsupported file type '{source.suffix}'. "
|
||||
"This skill only converts .pdf files.",
|
||||
file=sys.stderr,
|
||||
)
|
||||
return EXIT_INVALID_INPUT
|
||||
|
||||
MarkItDown = _import_markitdown()
|
||||
fitz = _import_fitz()
|
||||
md = MarkItDown()
|
||||
|
||||
if source.is_file():
|
||||
dest_dir = Path(args.output) if args.output else source.parent / source.stem
|
||||
return EXIT_OK if convert_one(md, fitz, source, dest_dir) else EXIT_CONVERSION_FAILED
|
||||
|
||||
# Directory / batch mode
|
||||
pdf_files, skipped = find_pdf_files(source, args.recursive)
|
||||
if skipped:
|
||||
print(f"NOTE: skipped {skipped} non-.pdf file(s) in {source}")
|
||||
if not pdf_files:
|
||||
print(f"ERROR: No .pdf files found under {source}", file=sys.stderr)
|
||||
return EXIT_INVALID_INPUT
|
||||
|
||||
out_dir = Path(args.output) if args.output else None
|
||||
success_count = 0
|
||||
for pdf_path in pdf_files:
|
||||
if out_dir is not None:
|
||||
rel = pdf_path.relative_to(source)
|
||||
dest_dir = out_dir / rel.parent / pdf_path.stem
|
||||
else:
|
||||
dest_dir = pdf_path.parent / pdf_path.stem
|
||||
if convert_one(md, fitz, pdf_path, dest_dir):
|
||||
success_count += 1
|
||||
|
||||
total = len(pdf_files)
|
||||
print(f"\nConverted {success_count}/{total} file(s).")
|
||||
return EXIT_OK if success_count == total else EXIT_CONVERSION_FAILED
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
@@ -0,0 +1,2 @@
|
||||
markitdown[pdf]>=0.1.0
|
||||
pymupdf>=1.24.0
|
||||
@@ -0,0 +1,129 @@
|
||||
---
|
||||
name: convert-word-to-md
|
||||
description: 'Converts Word (.docx) documents into Markdown so their contents can be accurately analyzed, summarized, searched, or extracted from. Use this skill whenever the user shares, references, or asks about a .docx file — even if they don''t say "convert" or "markdown" explicitly. This includes requests to "read", "summarize", "review", "extract data from", "compare", or "analyze" a Word document, resume, report, contract, or proposal. Always run the bundled conversion script to produce Markdown first; do not attempt to parse .docx content directly or write ad-hoc conversion code. Also use this skill for batch requests involving a whole folder of Word documents. IMPORTANT: When the user references a folder or set of documents containing multiple file types (.pdf, .docx, .xlsx), invoke ALL three sibling skills — convert-pdf-to-md, convert-word-to-md, and convert-excel-to-md — so no file type is silently skipped.'
|
||||
---
|
||||
|
||||
# Convert Word to Markdown
|
||||
|
||||
## When to use this skill
|
||||
|
||||
Trigger this skill any time there is a `.docx` file that needs to be
|
||||
understood or processed — for example, a user attaches a Word document and
|
||||
asks questions about it, wants a summary, wants specific data pulled out, or
|
||||
wants multiple Word documents in a folder processed together. Word's native
|
||||
`.docx` format is a zipped XML bundle that is not reliably readable as plain
|
||||
text, so always convert it to Markdown first using the script in this
|
||||
skill rather than trying to open or parse the file directly.
|
||||
|
||||
This skill only supports `.docx`. If asked to convert a legacy `.doc` file,
|
||||
tell the user it isn't supported and ask them to re-save it as `.docx`
|
||||
(Word: File > Save As > Word Document (.docx)) first.
|
||||
|
||||
**Mixed file types:** When the user references a folder or set of documents
|
||||
containing multiple supported file types (`.pdf`, `.docx`, `.xlsx`), this
|
||||
skill handles only `.docx` files. The agent MUST also invoke the sibling
|
||||
skills in parallel:
|
||||
- `convert-pdf-to-md` for any `.pdf` files
|
||||
- `convert-excel-to-md` for any `.xlsx` files
|
||||
|
||||
Never process a folder and silently skip a supported file type. All three
|
||||
skills must be invoked together when mixed types are present.
|
||||
|
||||
## Setup (once per environment)
|
||||
|
||||
Before the first conversion in a given environment, follow
|
||||
[`references/setup.md`](references/setup.md) step by step to ensure Python,
|
||||
pip, and the `markitdown` package are installed. Do this proactively rather
|
||||
than guessing whether the environment is ready — the script itself will
|
||||
also fail with a clear pointer back to that file if `markitdown` turns out
|
||||
to be missing, so it's safe to just try the conversion first if you're
|
||||
reasonably confident setup was already done.
|
||||
|
||||
## Usage
|
||||
|
||||
The conversion script lives at `scripts/convert_word_to_md.py`.
|
||||
|
||||
**Output structure:** MarkItDown embeds images as a truncated `data:image/png;base64...` URI
|
||||
placeholder (not real image data), so the script
|
||||
extracts real images directly from the `.docx` and writes a self-contained
|
||||
folder per document instead of a single loose `.md` file:
|
||||
|
||||
```
|
||||
<name>/
|
||||
img/
|
||||
img001.<ext>
|
||||
img002.<ext>
|
||||
...
|
||||
<name>.md (image references are relative: img/imgNNN.ext)
|
||||
```
|
||||
|
||||
If the document has no embedded images, no `img/` folder is created.
|
||||
|
||||
**Single file:**
|
||||
|
||||
```powershell
|
||||
# Windows
|
||||
python scripts\convert_word_to_md.py "C:\path\to\document.docx"
|
||||
```
|
||||
|
||||
```bash
|
||||
# macOS / Linux
|
||||
python scripts/convert_word_to_md.py "/path/to/document.docx"
|
||||
```
|
||||
|
||||
This creates a `document\` folder next to the source file (containing
|
||||
`document.md` and, if present, `document\img\`). To control the destination
|
||||
folder explicitly:
|
||||
|
||||
```powershell
|
||||
python scripts\convert_word_to_md.py "C:\path\to\document.docx" -o "C:\path\to\output_folder"
|
||||
```
|
||||
|
||||
**A folder of Word documents (batch mode):**
|
||||
|
||||
```powershell
|
||||
python scripts\convert_word_to_md.py "C:\path\to\folder"
|
||||
```
|
||||
|
||||
Add `--recursive` to also include subfolders:
|
||||
|
||||
```powershell
|
||||
python scripts\convert_word_to_md.py "C:\path\to\folder" --recursive
|
||||
```
|
||||
|
||||
Each `.docx` found gets its own `<name>\` output folder next to it by
|
||||
default. Pass `-o "C:\path\to\output_parent"` to collect all the generated
|
||||
`<name>\` folders under a separate parent directory instead (subfolder
|
||||
structure is preserved when combined with `--recursive`).
|
||||
|
||||
After conversion, read the resulting `.md` file(s) to perform the actual
|
||||
analysis the user asked for — the script's job is only to produce accurate
|
||||
Markdown (and images), not to interpret the content.
|
||||
|
||||
## Deciding where output goes
|
||||
|
||||
**Default — always output next to the source file.** The `<name>/` folder
|
||||
is created in the same directory as the source `.docx`. This is the required
|
||||
default for every case. Do NOT override it unless the user explicitly asks
|
||||
for a different location.
|
||||
|
||||
**Only use `-o` when** the user explicitly provides an output path (e.g.,
|
||||
"save the output to `C:\output`", "put the results in `D:\work`"). Do NOT
|
||||
pass `-o` based on the agent's current working directory, the session state
|
||||
folder, or any implied location.
|
||||
|
||||
**If the source file path cannot be fully resolved** — for example, the
|
||||
user provides only a filename with no directory, or the path is ambiguous —
|
||||
use `ask_user` to confirm the full absolute path before running the
|
||||
conversion. Never guess or assume the directory.
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
| Symptom | Likely cause | Fix |
|
||||
|---|---|---|
|
||||
| `ModuleNotFoundError: No module named 'markitdown'` / exit code 2 | MarkItDown not installed | Follow `references/setup.md` |
|
||||
| `ERROR: Unsupported file type '.doc'` / exit code 3 | Legacy `.doc`, not `.docx` | Ask the user to re-save as `.docx` |
|
||||
| `ERROR: Input path not found` / exit code 3 | Wrong path, or file moved | Confirm the correct path with the user |
|
||||
| `FAILED <file> -> ...` in batch output | That specific file is corrupt, password-protected, or otherwise unreadable | Report which file(s) failed; other files in the batch still succeed |
|
||||
| `NOTE: skipped N non-.docx file(s)` | Folder contains non-Word files | Expected — those files are intentionally ignored |
|
||||
| `WARNING: found N image placeholder(s) ... but extracted M image file(s)` | Mismatch between MarkItDown's placeholder count and images found in `word/media/` (unusual/malformed docx) | Placeholders are left unreplaced rather than risk wrong images; inspect the source file's media manually if images are needed |
|
||||
@@ -0,0 +1,66 @@
|
||||
# Environment Setup for convert-word-to-md
|
||||
|
||||
Follow these steps exactly, in order, before running `scripts/convert_word_to_md.py`
|
||||
for the first time in a given environment. Don't skip steps or improvise
|
||||
alternatives — they're written to be deterministic and safe to re-run.
|
||||
|
||||
## 1. Check Python is available (3.10+)
|
||||
|
||||
```powershell
|
||||
python --version
|
||||
```
|
||||
|
||||
- If this fails (command not found), install Python 3.10 or newer:
|
||||
- Windows: `winget install --id Python.Python.3.12 -e`
|
||||
- macOS: `brew install python@3.12`
|
||||
- Linux (Debian/Ubuntu): `sudo apt-get update && sudo apt-get install -y python3 python3-pip python-is-python3`
|
||||
- If the reported version is older than 3.10, install a newer Python using
|
||||
the same command above (MarkItDown requires 3.10+).
|
||||
|
||||
## 2. Check pip is available
|
||||
|
||||
```powershell
|
||||
python -m pip --version
|
||||
```
|
||||
|
||||
- If this fails, bootstrap pip:
|
||||
|
||||
```powershell
|
||||
python -m ensurepip --upgrade
|
||||
```
|
||||
|
||||
## 3. Install MarkItDown with Word (.docx) support
|
||||
|
||||
Use the `scripts/requirements.txt` file bundled with this skill to install a pinned,
|
||||
known-good version of the dependency:
|
||||
|
||||
```powershell
|
||||
python -m pip install -r scripts/requirements.txt
|
||||
```
|
||||
|
||||
This pulls in `markitdown[docx]` (MarkItDown's Word conversion dependency, which
|
||||
includes `mammoth` for `.docx` file parsing). No extra package is needed — this
|
||||
skill's script uses MarkItDown's built-in Word converter.
|
||||
|
||||
## 4. Verify the install
|
||||
|
||||
```powershell
|
||||
python -c "from markitdown import MarkItDown; print('markitdown OK')"
|
||||
```
|
||||
|
||||
Expect to see `markitdown OK` printed with no errors. If you see
|
||||
`ModuleNotFoundError: No module named 'markitdown'`, repeat step 3 — pip may
|
||||
be installing into a different Python environment than the one being
|
||||
invoked (check `python -m pip --version` shows the same path as `python
|
||||
--version`'s interpreter).
|
||||
|
||||
## Notes
|
||||
|
||||
- This setup only needs to be done once per environment/virtual environment,
|
||||
not once per conversion.
|
||||
- `convert_word_to_md.py` itself also checks for `markitdown` at startup and
|
||||
prints a pointer back to this file if it's missing, so re-running setup is
|
||||
safe and idempotent.
|
||||
- Only `.docx` is supported by this skill. Legacy binary `.doc` files are
|
||||
out of scope — ask the user to re-save the file as `.docx` (e.g., via
|
||||
Word's "Save As") if one is encountered.
|
||||
@@ -0,0 +1,301 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Convert Word (.docx) documents to Markdown using Microsoft's MarkItDown,
|
||||
with embedded images extracted to real files (MarkItDown only emits a
|
||||
truncated `data:image/...;base64...` placeholder, not real image data).
|
||||
|
||||
Usage:
|
||||
python convert_word_to_md.py <input> [-o OUTPUT] [--recursive]
|
||||
|
||||
<input> may be either:
|
||||
- a path to a single .docx file, or
|
||||
- a path to a directory (batch mode: every .docx file directly inside it
|
||||
is converted; pass --recursive to also descend into subdirectories).
|
||||
|
||||
Output:
|
||||
For each source .docx (named "<name>.docx"), a folder is created
|
||||
containing the Markdown and its images, in this layout:
|
||||
|
||||
<name>/
|
||||
img/
|
||||
img001.<ext>
|
||||
img002.<ext>
|
||||
...
|
||||
<name>.md (image references are relative: img/imgNNN.ext)
|
||||
|
||||
- Single file mode: the "<name>/" folder is created next to the source
|
||||
file, or at -o/--output (treated as the exact destination folder) if
|
||||
given.
|
||||
- Batch/directory mode: a "<name>/" folder is created next to each source
|
||||
file, or under -o/--output (treated as a parent directory, created if
|
||||
missing) if given, preserving relative subfolder structure when
|
||||
--recursive is used.
|
||||
- If a document has no embedded images, no "img/" folder is created.
|
||||
|
||||
Exit codes:
|
||||
0 - all requested conversions succeeded
|
||||
1 - one or more conversions failed (partial success in batch mode)
|
||||
2 - required dependency ("markitdown") is not installed
|
||||
3 - invalid input (path not found, or single-file input is not .docx)
|
||||
"""
|
||||
import argparse
|
||||
import re
|
||||
import shutil
|
||||
import sys
|
||||
import zipfile
|
||||
from pathlib import Path
|
||||
from xml.etree import ElementTree as ET
|
||||
|
||||
EXIT_OK = 0
|
||||
EXIT_CONVERSION_FAILED = 1
|
||||
EXIT_MISSING_DEPENDENCY = 2
|
||||
EXIT_INVALID_INPUT = 3
|
||||
|
||||
_W_NS = "http://schemas.openxmlformats.org/wordprocessingml/2006/main"
|
||||
_R_NS = "http://schemas.openxmlformats.org/officeDocument/2006/relationships"
|
||||
_REL_NS = "http://schemas.openxmlformats.org/package/2006/relationships"
|
||||
|
||||
# MarkItDown embeds images as a literal truncated placeholder, e.g.
|
||||
#  -- NOT real base64 data. This pattern
|
||||
# matches that placeholder so it can be swapped for a real relative path.
|
||||
_PLACEHOLDER_IMAGE_RE = re.compile(
|
||||
r'!\[([^\]]*)\]\(data:image/[a-zA-Z0-9.+-]+;base64[^)]*\)'
|
||||
)
|
||||
|
||||
|
||||
def _import_markitdown():
|
||||
"""Import MarkItDown, failing with a clear, actionable message if absent."""
|
||||
try:
|
||||
from markitdown import MarkItDown
|
||||
return MarkItDown
|
||||
except ImportError:
|
||||
print(
|
||||
"ERROR: The 'markitdown' package is not installed.\n"
|
||||
"See references/setup.md for this skill, or run:\n"
|
||||
' pip install "markitdown[docx]"',
|
||||
file=sys.stderr,
|
||||
)
|
||||
sys.exit(EXIT_MISSING_DEPENDENCY)
|
||||
|
||||
|
||||
def _document_order_media(docx_path: Path):
|
||||
"""Return [(rel_id, media_zip_path), ...] in the order images appear in
|
||||
word/document.xml (via r:embed / r:id), resolved through
|
||||
word/_rels/document.xml.rels. Returns [] if the document has no body
|
||||
part or no images (e.g. malformed docx falls back gracefully)."""
|
||||
try:
|
||||
with zipfile.ZipFile(docx_path) as z:
|
||||
if "word/document.xml" not in z.namelist() or \
|
||||
"word/_rels/document.xml.rels" not in z.namelist():
|
||||
return []
|
||||
rels_xml = z.read("word/_rels/document.xml.rels")
|
||||
doc_xml = z.read("word/document.xml")
|
||||
except (zipfile.BadZipFile, KeyError, OSError):
|
||||
return []
|
||||
|
||||
try:
|
||||
rels_root = ET.fromstring(rels_xml)
|
||||
doc_root = ET.fromstring(doc_xml)
|
||||
except ET.ParseError:
|
||||
return []
|
||||
|
||||
rel_map = {}
|
||||
for rel in rels_root.findall(f"{{{_REL_NS}}}Relationship"):
|
||||
rel_map[rel.get("Id")] = rel.get("Target")
|
||||
|
||||
ordered_rel_ids = []
|
||||
for elem in doc_root.iter():
|
||||
tag = elem.tag.rsplit("}", 1)[-1]
|
||||
if tag == "blip":
|
||||
rid = elem.get(f"{{{_R_NS}}}embed")
|
||||
elif tag == "imagedata":
|
||||
rid = elem.get(f"{{{_R_NS}}}id")
|
||||
else:
|
||||
rid = None
|
||||
if rid:
|
||||
ordered_rel_ids.append(rid)
|
||||
ordered_media = []
|
||||
for rid in ordered_rel_ids:
|
||||
target = rel_map.get(rid)
|
||||
if not target or "media/" not in target:
|
||||
continue
|
||||
import posixpath
|
||||
media_path = (
|
||||
target.lstrip("/")
|
||||
if target.startswith("/")
|
||||
else posixpath.normpath(
|
||||
target if target.startswith("word/") else posixpath.join("word", target)
|
||||
)
|
||||
)
|
||||
ordered_media.append((rid, media_path))
|
||||
return ordered_media
|
||||
|
||||
|
||||
def _extract_images(docx_path: Path, img_dir: Path):
|
||||
"""Extract embedded images from docx_path into img_dir as img001.ext,
|
||||
img002.ext, ... in document order. Returns the list of written filenames
|
||||
(relative to img_dir), in that same order."""
|
||||
ordered_media = _document_order_media(docx_path)
|
||||
if not ordered_media:
|
||||
return []
|
||||
|
||||
written = []
|
||||
with zipfile.ZipFile(docx_path) as z:
|
||||
names_in_zip = set(z.namelist())
|
||||
for idx, (rid, media_path) in enumerate(ordered_media, start=1):
|
||||
if media_path not in names_in_zip:
|
||||
print(f"WARNING: {media_path} (rel {rid}) not found in {docx_path}", file=sys.stderr)
|
||||
continue
|
||||
ext = Path(media_path).suffix.lstrip(".").lower() or "bin"
|
||||
if ext == "jpg":
|
||||
ext = "jpeg"
|
||||
out_name = f"img{idx:03d}.{ext}"
|
||||
img_dir.mkdir(parents=True, exist_ok=True)
|
||||
(img_dir / out_name).write_bytes(z.read(media_path))
|
||||
written.append(out_name)
|
||||
return written
|
||||
|
||||
|
||||
def _rewrite_image_refs(markdown_text: str, image_files) -> str:
|
||||
"""Replace MarkItDown's truncated base64 image placeholders with real
|
||||
relative img/imgNNN.ext references, in left-to-right order. If the
|
||||
counts don't match (unexpected), the placeholders are left as-is rather
|
||||
than risk mismatched references."""
|
||||
matches = list(_PLACEHOLDER_IMAGE_RE.finditer(markdown_text))
|
||||
if not matches:
|
||||
return markdown_text
|
||||
if len(matches) != len(image_files):
|
||||
print(
|
||||
f"WARNING: found {len(matches)} image placeholder(s) in markdown but "
|
||||
f"extracted {len(image_files)} image file(s); leaving placeholders "
|
||||
"unreplaced to avoid mismatched references.",
|
||||
file=sys.stderr,
|
||||
)
|
||||
return markdown_text
|
||||
|
||||
counter = {"i": 0}
|
||||
|
||||
def _replace(m):
|
||||
name = image_files[counter["i"]]
|
||||
counter["i"] += 1
|
||||
return f""
|
||||
|
||||
return _PLACEHOLDER_IMAGE_RE.sub(_replace, markdown_text)
|
||||
|
||||
|
||||
def convert_one(md, source: Path, dest_dir: Path) -> bool:
|
||||
"""Convert a single .docx file to a "<name>/" folder containing the
|
||||
Markdown file and an "img/" folder of extracted images. Returns True on
|
||||
success."""
|
||||
try:
|
||||
result = md.convert(str(source))
|
||||
except ImportError as exc:
|
||||
print(
|
||||
f"ERROR: A required dependency for converting '{source.name}' is not installed.\n"
|
||||
f" {exc}\n"
|
||||
"See references/setup.md for this skill, or run:\n"
|
||||
' pip install "markitdown[docx]"',
|
||||
file=sys.stderr,
|
||||
)
|
||||
sys.exit(EXIT_MISSING_DEPENDENCY)
|
||||
except Exception as exc: # noqa: BLE001 - surface any conversion error
|
||||
print(f"FAILED {source} -> {exc}", file=sys.stderr)
|
||||
return False
|
||||
|
||||
try:
|
||||
if dest_dir.exists():
|
||||
shutil.rmtree(dest_dir)
|
||||
dest_dir.mkdir(parents=True, exist_ok=True)
|
||||
image_files = _extract_images(source, dest_dir / "img")
|
||||
text = _rewrite_image_refs(result.text_content, image_files)
|
||||
md_path = dest_dir / f"{source.stem}.md"
|
||||
md_path.write_text(text, encoding="utf-8")
|
||||
except OSError as exc:
|
||||
print(f"FAILED {source} -> could not write output in {dest_dir}: {exc}", file=sys.stderr)
|
||||
return False
|
||||
|
||||
img_note = f", {len(image_files)} image(s)" if image_files else ""
|
||||
print(f"OK {source} -> {md_path}{img_note}")
|
||||
return True
|
||||
|
||||
|
||||
def find_docx_files(root: Path, recursive: bool):
|
||||
"""Return (docx_files, skipped_count) for files directly/recursively under root."""
|
||||
pattern_iter = root.rglob("*") if recursive else root.iterdir()
|
||||
docx_files = []
|
||||
skipped = 0
|
||||
for entry in pattern_iter:
|
||||
if entry.is_dir():
|
||||
continue
|
||||
if entry.suffix.lower() == ".docx":
|
||||
docx_files.append(entry)
|
||||
else:
|
||||
skipped += 1
|
||||
return sorted(docx_files), skipped
|
||||
|
||||
|
||||
def main() -> int:
|
||||
parser = argparse.ArgumentParser(description=__doc__.splitlines()[0])
|
||||
parser.add_argument("input", help="Path to a .docx file or a directory of .docx files")
|
||||
parser.add_argument(
|
||||
"-o", "--output",
|
||||
help=(
|
||||
"Destination folder for the '<name>/' output (single-file mode), "
|
||||
"or parent directory under which each '<name>/' output folder is "
|
||||
"created (batch mode)"
|
||||
),
|
||||
)
|
||||
parser.add_argument(
|
||||
"--recursive", action="store_true",
|
||||
help="When input is a directory, also search subdirectories",
|
||||
)
|
||||
args = parser.parse_args()
|
||||
|
||||
#MarkItDown = _import_markitdown()
|
||||
#md = MarkItDown()
|
||||
|
||||
source = Path(args.input)
|
||||
if not source.exists():
|
||||
print(f"ERROR: Input path not found: {source}", file=sys.stderr)
|
||||
return EXIT_INVALID_INPUT
|
||||
|
||||
if source.is_file() and source.suffix.lower() != ".docx":
|
||||
print(
|
||||
f"ERROR: Unsupported file type '{source.suffix}'. "
|
||||
"This skill only converts .docx files.",
|
||||
file=sys.stderr,
|
||||
)
|
||||
return EXIT_INVALID_INPUT
|
||||
|
||||
MarkItDown = _import_markitdown()
|
||||
md = MarkItDown()
|
||||
|
||||
if source.is_file():
|
||||
dest_dir = Path(args.output) if args.output else source.parent / source.stem
|
||||
return EXIT_OK if convert_one(md, source, dest_dir) else EXIT_CONVERSION_FAILED
|
||||
|
||||
# Directory / batch mode
|
||||
docx_files, skipped = find_docx_files(source, args.recursive)
|
||||
if skipped:
|
||||
print(f"NOTE: skipped {skipped} non-.docx file(s) in {source}")
|
||||
if not docx_files:
|
||||
print(f"ERROR: No .docx files found under {source}", file=sys.stderr)
|
||||
return EXIT_INVALID_INPUT
|
||||
|
||||
out_dir = Path(args.output) if args.output else None
|
||||
success_count = 0
|
||||
for docx_path in docx_files:
|
||||
if out_dir is not None:
|
||||
rel = docx_path.relative_to(source)
|
||||
dest_dir = out_dir / rel.parent / docx_path.stem
|
||||
else:
|
||||
dest_dir = docx_path.parent / docx_path.stem
|
||||
if convert_one(md, docx_path, dest_dir):
|
||||
success_count += 1
|
||||
|
||||
total = len(docx_files)
|
||||
print(f"\nConverted {success_count}/{total} file(s).")
|
||||
return EXIT_OK if success_count == total else EXIT_CONVERSION_FAILED
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
@@ -0,0 +1 @@
|
||||
markitdown[docx]>=0.1.0
|
||||
Reference in New Issue
Block a user