Analyze Codebase Workflow

Survey an arbitrary repository to auto-detect data flows, file I/O, and script dependencies, then produce a structured annotation plan for manual refinement.

When to Use

• Onboarding onto an unfamiliar codebase and need to understand data flow
• Starting putior integration in a project that has no PUT annotations yet
• Auditing an existing project's data pipeline before documentation
• Preparing an annotation plan before running annotate-source-files

Inputs

• Required: Path to the repository or source directory to analyze
• Optional: Specific subdirectories to focus on (default: entire repo)
• Optional: Languages to include or exclude (default: all detected)
• Optional: Detection scope: inputs only, outputs only, or both (default: both + dependencies)

Procedure

Step 1: Survey Repository Structure

Identify source files and their languages to understand what putior can analyze.

library(putior)

# List all supported languages and their extensions
list_supported_languages()
list_supported_languages(detection_only = TRUE)  # Only languages with auto-detection

# Get supported extensions
exts <- get_supported_extensions()

Use file listing to understand repo composition:

# Count files by extension in the target directory
find /path/to/repo -type f | sed 's/.*\.//' | sort | uniq -c | sort -rn | head -20

Expected: A list of file extensions present in the repo, with counts. Map these against getsupportedextensions() to know coverage.

On failure: If the repo has no files matching supported extensions, putior cannot auto-detect workflows. Consider whether the language is supported but files use non-standard extensions.

Step 2: Check Language Detection Coverage

For each detected language, verify auto-detection pattern availability.

# Check which languages have auto-detection patterns (18 languages, 902 patterns)
detection_langs <- list_supported_languages(detection_only = TRUE)
cat("Languages with auto-detection:\n")
print(detection_langs)

# Get pattern counts for specific languages found in the repo
for (lang in c("r", "python", "javascript", "sql", "dockerfile", "makefile")) {
  patterns <- get_detection_patterns(lang)
  cat(sprintf("%s: %d input, %d output, %d dependency patterns\n",
    lang,
    length(patterns$input),
    length(patterns$output),
    length(patterns$dependency)
  ))
}

Expected: Pattern counts printed for each language. R has 124 patterns, Python 159, JavaScript 71, etc.

On failure: If a language returns no patterns, it supports manual annotations but not auto-detection. Plan to annotate those files manually.

Step 3: Run Auto-Detection

Execute put_auto() on the target directory to discover workflow elements.

# Full auto-detection
workflow <- put_auto("./src/",
  detect_inputs = TRUE,
  detect_outputs = TRUE,
  detect_dependencies = TRUE
)

# Exclude build scripts and test helpers from scanning
workflow <- put_auto("./src/",
  detect_inputs = TRUE,
  detect_outputs = TRUE,
  detect_dependencies = TRUE,
  exclude = c("build-", "test_helper")
)

# View detected workflow nodes
print(workflow)

# Check node count
cat(sprintf("Detected %d workflow nodes\n", nrow(workflow)))

For large repos, analyze subdirectories incrementally:

# Analyze specific subdirectories
etl_workflow <- put_auto("./src/etl/")
api_workflow <- put_auto("./src/api/")

Expected: A data frame with columns including id, label, input, output, source_file. Each row represents a detected workflow step.

On failure: If the result is empty, the source files may not contain recognizable I/O patterns. Try enabling debug logging: workflow <- putauto("./src/", loglevel = "DEBUG") to see which files are scanned and which patterns match.

Step 4: Generate Initial Diagram

Visualize the auto-detected workflow to assess coverage and identify gaps.

# Generate diagram from auto-detected workflow
cat(put_diagram(workflow, theme = "github"))

# With source file info for traceability
cat(put_diagram(workflow, show_source_info = TRUE))

# Save to file for review
writeLines(put_diagram(workflow, theme = "github"), "workflow-auto.md")

Expected: A Mermaid flowchart showing detected nodes connected by data flow edges. Nodes should be labeled with meaningful function/file names.

On failure: If the diagram shows disconnected nodes, the auto-detection found I/O patterns but couldn't infer connections. This is normal — connections are derived from matching output filenames to input filenames. The annotation plan (next step) will address gaps.

Step 5: Produce Annotation Plan

Generate a structured plan documenting what was found and what needs manual annotation.

# Generate annotation suggestions
put_generate("./src/", style = "single")

# For multiline style (more readable for complex workflows)
put_generate("./src/", style = "multiline")

# Copy suggestions to clipboard for easy pasting
put_generate("./src/", output = "clipboard")

Document the plan with coverage assessment:

## Annotation Plan

### Auto-Detected (no manual work needed)
- `src/etl/extract.R` — 3 inputs, 2 outputs detected
- `src/etl/transform.py` — 1 input, 1 output detected

### Needs Manual Annotation
- `src/api/handler.js` — Language supported but no I/O patterns matched
- `src/config/setup.sh` — Only 12 shell patterns; complex logic missed

### Not Supported
- `src/legacy/process.f90` — Fortran not in detection languages

### Recommended Connections
- extract.R output `data.csv` → transform.py input `data.csv` (auto-linked)
- transform.py output `clean.parquet` → load.R input (needs annotation)

Expected: A clear plan separating auto-detected files from those needing manual annotation, with specific recommendations for each file.

On failure: If put_generate() produces no output, ensure the directory path is correct and contains source files in supported languages.

Validation

• put_auto() executes without errors on the target directory
• Detected workflow has at least one node (unless repo has no recognizable I/O)
• put_diagram() produces valid Mermaid code from the auto-detected workflow
• put_generate() produces annotation suggestions for files with detected patterns
• Annotation plan document created with coverage assessment

Common Pitfalls

• Scanning too broadly: Running putauto(".") on a repo root may include nodemodules/, .git/, venv/, etc. Target specific source directories.
• Expecting full coverage: Auto-detection finds file I/O and library calls, not business logic. A 40-60% coverage rate is typical; the rest needs manual annotation.
• Ignoring dependencies: The detect_dependencies = TRUE flag catches source(), import, require() calls that link scripts together. Disabling it loses cross-file connections.
• Language mismatch: Files with non-standard extensions (e.g., .R vs .r, .jsx vs .js) may not be detected. Use getcommentprefix() to check if an extension is recognized. Note that extensionless files like Dockerfile and Makefile are supported via exact filename matching.
• Large repos: For repos with 100+ source files, analyze by module/directory to keep diagrams readable.

Related Skills

• install-putior — prerequisite: putior must be installed first
• annotate-source-files — next step: add manual annotations based on the plan
• generate-workflow-diagram — generate final diagram after annotation is complete
• configure-putior-mcp — use MCP tools for interactive analysis sessions