CodeWeaver: Generate Markdown Documentation from Your Codebase

CodeWeaver is a command-line tool that transforms your codebase into a single, navigable Markdown document. It recursively scans a directory, creating a tree-like representation of your project's file structure and embedding the content of each file within markdown code blocks. This simplifies codebase sharing, documentation, and integration with AI/ML tools by providing a consolidated, readable Markdown output.

The output for the current repository can be found here.

Key Features

Comprehensive Codebase Documentation: Generates a Markdown file outlining your project's directory and file structure in a clear, tree-like format.
Code Content Inclusion: Embeds the complete content of each file within the Markdown document, using code blocks based on file extensions.
Flexible Path Filtering: Uses regular expressions to define include and / or ignore patterns, giving you precise control over which files are included.
Optional Path Logging: Saves lists of included and excluded file paths to separate files for detailed tracking.
Clipboard Integration: Optionally copies the generated Markdown to the clipboard for easy pasting.
Simple CLI: A straightforward command-line interface with intuitive options.

Installation

Using go install (Recommended):

Requires Go 1.18 or later.

go install github.com/tesserato/CodeWeaver@latest

To install a specific version:

go install github.com/tesserato/CodeWeaver@vX.Y.Z  # Replace X.Y.Z with the desired version

From Pre-built Executables:

Download the appropriate executable for your operating system from the releases page.

After downloading, make the executable:

chmod +x codeweaver  # On Linux/macOS

Usage

codeweaver [options]

For help:

codeweaver -h

Options:

Option	Description	Default Value
`-input <directory>`	The root directory to scan.	`.` (current directory)
`-output <filename>`	The name of the output Markdown file.	`codebase.md`
`-ignore "<regex patterns>"`	Comma-separated list of regular expressions for paths to exclude. Example: `\.git.,node_modules,.log`	`\.git.*`
`-include "<regex patterns>"`	Comma-separated list of regular expressions. Only paths matching these are included. Example: `\.go$,\.md$`	None
`-included-paths-file <filename>`	Saves the list of included paths to this file.	None
`-excluded-paths-file <filename>`	Saves the list of excluded paths to this file.	None
`-clipboard`	Copies the generated Markdown to the clipboard.	`false`
`-version`	Displays the version and exits.
`-help`	Displays this help message and exits.

Understanding -include and -ignore

These flags control which files and directories are included in the generated documentation.

-ignore (Blacklist): Excludes files/directories matching any of the provided regular expressions.
-include (Whitelist): Only includes files/directories matching at least one of the provided regular expressions. If -include is used, everything else is excluded by default.

Behavior Table:

`-ignore`	`-include`	Behavior
No	No	Includes all files/directories except the input directory itself (`.`).
Yes	No	Excludes files/directories matching `-ignore`; includes everything else.
No	Yes	Only includes files/directories matching `-include`. Everything else is excluded.
Yes	Yes	Includes files/directories that match at least one `-include` pattern AND do not match any `-ignore` pattern. `-include` creates a whitelist, and `-ignore` filters it.

Examples

1. Basic Usage:

codeweaver

Creates codebase.md in the current directory, documenting the structure and content (excluding paths matching the default ignore pattern \.git.*).

2. Different Input/Output:

codeweaver -input=my_project -output=project_docs.md

Processes my_project and saves the output to project_docs.md.

3. Ignoring Files/Directories:

codeweaver -ignore="\.log,temp,build"

Excludes files/directories named .log, temp, or build.

4. Including Only Specific Files:

codeweaver -include="\.go$,\.md$"

Includes only Go (.go) and Markdown (.md) files.

5. Combining include and ignore:

codeweaver -include="\.go$,\.md$" -ignore="vendor,test"

Includes Go and Markdown files, except those with "vendor" or "test" in their paths.

6. Saving Included/Excluded Paths:

codeweaver -ignore="node_modules" -included-paths-file=included.txt -excluded-paths-file=excluded.txt

Creates codebase.md, saves included paths to included.txt, and excluded paths to excluded.txt.

7. Copying to Clipboard:

codeweaver -clipboard

Creates codebase.md and copies its content to the clipboard.

8. Regex Examples:

.: Matches any single character.
*: Matches zero or more of the preceding character.
+: Matches one or more of the preceding character.
?: Matches zero or one of the preceding character.
[abc]: Matches any one of the characters inside the brackets.
[^abc]: Matches any character not inside the brackets.
[a-z]: Matches any character in the range a-z.
^: Matches the beginning of the string.
$: Matches the end of the string.
\.: Matches a literal dot (.). You need to escape it because . has special meaning in regex.
\|: Used for alternation (OR). e.g., a\|b matches either "a" or "b".
.*\.py[cod]$: matches python files that end with pyc, pyd or pyo.
.*\.pdf: matches PDF files.
(dir1\|dir2): matches dir1 or dir2

9. Complete example:

codeweaver -input=. -output=codebase.md -ignore="\.git.*,.+\.exe,codebase.md,excluded_paths.txt" -include="\.go$,\.md$,\.ps1$,\.yaml$,\.txt$,\.csv$" -excluded-paths-file="excluded_paths.txt" -clipboard

This command will:

Process the current directory (.).
Generate documentation and save it in codebase.md.
Exclude files matching .git.*, .+\.exe, the output file (codebase.md), and the file where the excluded paths will be saved.
Include only files with the extensions .go, .md, .ps1, .yaml, .txt, and .csv.
Save the list of excluded files in a file named excluded_paths.txt.
Copy the generated Markdown to the system clipboard.

Contributing

Contributions are welcome! Please open an issue or submit a pull request on the project's GitHub repository.

License

CodeWeaver is released under the MIT License.

Star History

Alternatives

This section lists tools with similar or overlapping functionality.

GitHub Repositories

Project	Stars
ai-context
bundle-codebases
code2prompt
code2text
codefetch
copcon
describe
feed-llm
files-to-prompt
ggrab
gitingest
gitpodcast
globcat.sh
grimoire
llmcat
RepoMix
RepoMix
techdocs
thisismy
yek
your-source-to-prompt
ingest
onefilellm
repo2file
clipsource