Babashka.fs File System Utilities Skill

Overview

The babashka.fs library is a comprehensive file system utility library for Clojure, designed for cross-platform file operations. It provides a clean, functional API for working with files, directories, and paths, built on top of Java's NIO.2 API while offering a more idiomatic Clojure interface.

When to use this skill:

When working with files and directories in Clojure/Babashka scripts
When you need cross-platform file system operations
When writing build tasks, file processing scripts, or automation tools
When you need to search, filter, or manipulate file systems programmatically

Setup and Requirements

Adding to your project

;; In deps.edn
{:deps {babashka/fs {:mvn/version "0.5.27"}}}

;; In your namespace
(ns my-script
  (:require [babashka.fs :as fs]))

Built-in to Babashka

The library is built into Babashka, so no additional dependencies are needed for bb scripts:

#!/usr/bin/env bb
(require '[babashka.fs :as fs])

(fs/directory? ".")  ; => true

Core Concepts

Path Objects

Most functions accept and return java.nio.file.Path objects, but also work with strings and other path-like objects. The library automatically coerces between types.

;; All of these work
(fs/exists? ".")
(fs/exists? (fs/path "."))
(fs/exists? (java.io.File. "."))

Cross-Platform Support

The library handles platform differences automatically, but provides utilities when you need platform-specific behavior:

;; Works on all platforms
(fs/path "dir" "subdir" "file.txt")

;; Convert to Unix-style paths (useful for Windows)
(fs/unixify "C:\\Users\\name\\file.txt")  ; => "C:/Users/name/file.txt"

Path Operations

Creating and Manipulating Paths

;; Create paths
(fs/path "dir" "subdir" "file.txt")              ; Join path components
(fs/file "dir" "subdir" "file.txt")              ; Alias for fs/path

;; Path properties
(fs/absolute? "/tmp/file.txt")                   ; true
(fs/relative? "dir/file.txt")                    ; true
(fs/hidden? ".hidden-file")                      ; Check if hidden

;; Path transformations
(fs/absolutize "relative/path")                  ; Convert to absolute
(fs/canonicalize "/tmp/../file.txt")             ; Resolve to canonical form
(fs/normalize "/tmp/./dir/../file.txt")          ; Normalize path

;; Path components
(fs/file-name "/path/to/file.txt")               ; "file.txt"
(fs/parent "/path/to/file.txt")                  ; "/path/to"
(fs/extension "file.txt")                        ; "txt"
(fs/split-ext "file.txt")                        ; ["file" "txt"]
(fs/strip-ext "file.txt")                        ; "file"

;; Path relationships
(fs/starts-with? "/foo/bar" "/foo")              ; true
(fs/ends-with? "/foo/bar.txt" "bar.txt")         ; true
(fs/relativize "/foo/bar" "/foo/bar/baz")        ; "baz"

;; Get all components
(fs/components "/path/to/file.txt")              ; Seq of path components

Working with Extensions

;; Get extension
(fs/extension "document.pdf")                     ; "pdf"
(fs/extension "archive.tar.gz")                   ; "gz"

;; Split filename and extension
(fs/split-ext "document.pdf")                     ; ["document" "pdf"]

;; Remove extension
(fs/strip-ext "document.pdf")                     ; "document"
(fs/strip-ext "archive.tar.gz")                   ; "archive.tar"

File and Directory Checks

;; Existence and type checks
(fs/exists? "file.txt")                           ; Does it exist?
(fs/directory? "path/to/dir")                     ; Is it a directory?
(fs/regular-file? "file.txt")                     ; Is it a regular file?
(fs/sym-link? "link")                             ; Is it a symbolic link?
(fs/hidden? ".hidden")                            ; Is it hidden?

;; Permission checks
(fs/readable? "file.txt")                         ; Can we read it?
(fs/writable? "file.txt")                         ; Can we write to it?
(fs/executable? "script.sh")                      ; Can we execute it?

;; Comparison
(fs/same-file? "file1.txt" "file2.txt")          ; Are they the same file?

Creating Files and Directories

;; Create directories
(fs/create-dir "new-dir")                         ; Create single directory
(fs/create-dirs "path/to/new/dir")               ; Create with parents

;; Create files
(fs/create-file "new-file.txt")                  ; Create empty file

;; Create temporary files/directories
(fs/create-temp-file)                            ; Creates temp file
(fs/create-temp-file {:prefix "data-"            ; Custom prefix/suffix
                       :suffix ".json"})
(fs/create-temp-dir)                             ; Creates temp directory
(fs/create-temp-dir {:prefix "workdir-"})

;; Create links
(fs/create-link "link-name" "target")            ; Hard link
(fs/create-sym-link "symlink" "target")          ; Symbolic link

;; Temporary directory context
(fs/with-temp-dir [tmp-dir {:prefix "work-"}]
  (println "Working in" (str tmp-dir))
  ;; Do work with tmp-dir
  ;; Directory automatically deleted after
  )

Reading and Writing Files

Reading Files

;; Read entire file
(slurp (fs/file "data.txt"))                     ; As string

;; Read lines
(with-open [rdr (io/reader (fs/file "data.txt"))]
  (doall (line-seq rdr)))

;; Or use fs helpers
(fs/read-all-lines "data.txt")                   ; Returns seq of lines
(fs/read-all-bytes "binary-file")                ; Returns byte array

Writing Files

;; Write text
(spit (fs/file "output.txt") "Hello, world!")

;; Write lines
(fs/write-lines "output.txt"
                ["Line 1" "Line 2" "Line 3"])
(fs/write-lines "output.txt"
                ["More lines"]
                {:append true})                   ; Append mode

;; Write bytes
(fs/write-bytes "output.bin" byte-array)
(fs/write-bytes "output.bin" byte-array
                {:append true})

Copying, Moving, and Deleting

;; Copy files
(fs/copy "source.txt" "dest.txt")                ; Copy file
(fs/copy "source.txt" "dest.txt"
         {:replace-existing true})               ; Overwrite if exists

;; Copy entire directory trees
(fs/copy-tree "source-dir" "dest-dir")           ; Recursive copy
(fs/copy-tree "source-dir" "dest-dir"
              {:replace-existing true})

;; Move/rename
(fs/move "old-name.txt" "new-name.txt")          ; Move or rename
(fs/move "file.txt" "other-dir/")                ; Move to directory

;; Delete
(fs/delete "file.txt")                           ; Delete single file
(fs/delete-if-exists "maybe-file.txt")           ; No error if missing
(fs/delete-tree "directory")                     ; Delete directory recursively

;; Delete on exit
(fs/delete-on-exit "temp-file.txt")              ; Delete when JVM exits

Listing and Traversing Directories

Simple Listing

;; List directory contents
(fs/list-dir ".")                                ; Seq of paths in directory
(fs/list-dir "." "*.txt")                        ; With glob pattern

;; List multiple directories
(fs/list-dirs ["dir1" "dir2"] "*.clj")           ; Combine results

;; Get directory stream (more efficient for large dirs)
(with-open [ds (fs/directory-stream "." "*.txt")]
  (doseq [path ds]
    (println path)))

Walking Directory Trees

;; Walk directory tree
(fs/walk-file-tree "."
  {:visit-file (fn [path attrs]
                 (println "File:" path)
                 :continue)
   :pre-visit-dir (fn [path attrs]
                    (println "Entering:" path)
                    :continue)
   :post-visit-dir (fn [path ex]
                     (println "Leaving:" path)
                     :continue)})

;; Common options
;; :max-depth - limit depth
;; :follow-links - follow symbolic links
;; :visit-file - called for each file
;; :pre-visit-dir - called before visiting directory
;; :post-visit-dir - called after visiting directory
;; :visit-file-failed - called when file access fails

Searching and Filtering: Glob and Match

Glob Patterns

The glob function is one of the most powerful features for finding files:

;; Find all Clojure files recursively
(fs/glob "." "**/*.clj")                         ; ** means recursive

;; Find files in current directory only
(fs/glob "." "*.txt")                            ; * means any characters

;; Multiple extensions
(fs/glob "." "**{.clj,.cljc,.cljs}")            ; Match multiple patterns

;; Complex patterns
(fs/glob "src" "**/test_*.clj")                 ; Test files anywhere
(fs/glob "." "data/*.{json,edn}")               ; JSON or EDN in data dir

;; Exclude patterns (use filter)
(->> (fs/glob "." "**/*.clj")
     (remove #(re-find #"/test/" (str %))))     ; Exclude test directories

;; Common glob patterns:
;; *     - matches any characters (not including /)
;; **    - matches any characters including /
;; ?     - matches single character
;; [abc] - matches any character in brackets
;; {a,b} - matches either a or b

Match with Regular Expressions

For more complex matching, use match:

;; Use regex for pattern matching
(fs/match "." "regex:.*\\.clj$" {:recursive true})

;; Or glob (explicit)
(fs/match "." "glob:**/*.clj" {:recursive true})

;; Options
(fs/match "src" "regex:test.*\\.clj"
          {:recursive true
           :hidden false                         ; Skip hidden files
           :follow-links false                   ; Don't follow symlinks
           :max-depth 5})                        ; Limit depth

Practical File Filtering Examples

;; Find large files
(->> (fs/glob "." "**/*")
     (filter fs/regular-file?)
     (filter #(> (fs/size %) (* 10 1024 1024)))  ; > 10MB
     (map str))

;; Find recently modified files
(->> (fs/glob "." "**/*.clj")
     (filter #(> (fs/file-time->millis (fs/last-modified-time %))
                 (- (System/currentTimeMillis)
                    (* 24 60 60 1000))))          ; Last 24 hours
     (map str))

;; Find files by owner (Unix)
(->> (fs/glob "/var/log" "*")
     (filter #(= "root" (str (fs/owner %))))
     (map str))

;; Find executable scripts
(->> (fs/glob "." "**/*.sh")
     (filter fs/executable?)
     (map str))

File Metadata and Attributes

;; File size
(fs/size "file.txt")                             ; Size in bytes

;; Timestamps
(fs/creation-time "file.txt")                    ; FileTime object
(fs/last-modified-time "file.txt")               ; FileTime object
(fs/set-last-modified-time "file.txt"
                           (fs/file-time 1234567890000))

;; Convert FileTime to millis
(fs/file-time->millis (fs/last-modified-time "file.txt"))
(fs/file-time->instant (fs/last-modified-time "file.txt"))

;; Create FileTime from millis
(fs/file-time 1234567890000)

;; Owner (Unix/Linux)
(fs/owner "file.txt")                            ; Returns owner object
(str (fs/owner "file.txt"))                      ; Owner name as string

;; POSIX permissions (Unix/Linux)
(fs/posix->str (fs/posix-file-permissions "file.txt"))  ; "rwxr-xr-x"
(fs/set-posix-file-permissions "file.txt"
                               (fs/str->posix "rwxr-xr-x"))

;; Check for modified files since anchor
(fs/modified-since "target" "src")               ; Files in src newer than target

Archive Operations (Zip)

;; Create zip archive
(fs/zip "archive.zip" "file1.txt")               ; Single file
(fs/zip "archive.zip" ["file1.txt"
                        "file2.txt"
                        "dir"])                   ; Multiple files/dirs

;; Zip with options
(fs/zip "archive.zip" "directory"
        {:root "directory"})                      ; Strip parent path

;; Extract zip archive
(fs/unzip "archive.zip" "output-dir")            ; Extract all

;; Extract with filter
(fs/unzip "archive.zip" "output-dir"
          {:extract-fn (fn [{:keys [name]}]
                         (re-find #"\\.txt$" name))})  ; Only .txt files

;; Manually work with zip entries
(fs/zip-path "archive.zip" "path/in/zip")        ; Access file in zip as path

System Paths and Utilities

;; User directories
(fs/home)                                        ; User home directory
(fs/temp-dir)                                    ; System temp directory
(fs/cwd)                                         ; Current working directory

;; XDG Base Directory Specification (Linux)
(fs/xdg-config-home)                             ; ~/.config
(fs/xdg-config-home "myapp")                     ; ~/.config/myapp
(fs/xdg-data-home)                               ; ~/.local/share
(fs/xdg-cache-home)                              ; ~/.cache
(fs/xdg-state-home)                              ; ~/.local/state

;; Executable paths
(fs/exec-paths)                                  ; All dirs in PATH
(fs/which "java")                                ; Find executable in PATH
(fs/which "git")                                 ; Returns path or nil

;; Find executable manually
(->> (fs/exec-paths)
     (mapcat #(fs/list-dir % "java*"))
     (filter fs/executable?)
     first)

Advanced Patterns and Best Practices

Safe File Operations with Error Handling

;; Check before operating
(when (fs/exists? "config.edn")
  (fs/copy "config.edn" "config.backup.edn"))

;; Use delete-if-exists for optional deletion
(fs/delete-if-exists "temp-file.txt")

;; Handle walk-file-tree errors
(fs/walk-file-tree "."
  {:visit-file-failed (fn [path ex]
                        (println "Failed to access:" path)
                        :skip-subtree)})

Working with Temporary Files

;; Pattern 1: with-temp-dir (automatic cleanup)
(fs/with-temp-dir [tmp-dir {:prefix "work-"}]
  (let [work-file (fs/path tmp-dir "data.txt")]
    (spit work-file "temporary data")
    (process-file work-file)))
;; tmp-dir automatically deleted here

;; Pattern 2: Manual temp file management
(let [tmp-file (fs/create-temp-file {:prefix "data-"
                                       :suffix ".json"})]
  (try
    (spit tmp-file (json/encode data))
    (process-file tmp-file)
    (finally
      (fs/delete tmp-file))))

;; Pattern 3: Delete on exit
(let [tmp-file (fs/create-temp-file)]
  (fs/delete-on-exit tmp-file)
  (spit tmp-file data)
  tmp-file)  ; File deleted when JVM exits

Efficient Directory Processing

;; Process large directories efficiently
(with-open [stream (fs/directory-stream "." "*.txt")]
  (doseq [path stream]
    (process-file path)))  ; Lazy processing, one at a time

;; Instead of realizing entire seq
(doseq [path (fs/list-dir "." "*.txt")]
  (process-file path))  ; Realizes all paths first

Cross-Platform Path Construction

;; Always use fs/path for joining - it handles separators
(fs/path "dir" "subdir" "file.txt")              ; Works everywhere

;; Don't manually concatenate with separators
;; BAD: (str "dir" "/" "subdir" "/" "file.txt")  ; Breaks on Windows

;; Convert Windows paths to Unix style when needed
(fs/unixify (fs/path "C:" "Users" "name"))       ; "C:/Users/name"

File Filtering Pipeline Pattern

;; Build reusable filters
(defn clojure-source? [path]
  (and (fs/regular-file? path)
       (re-find #"\.(clj|cljs|cljc)$" (str path))))

(defn recent? [days path]
  (let [cutoff (- (System/currentTimeMillis)
                  (* days 24 60 60 1000))]
    (> (fs/file-time->millis (fs/last-modified-time path)) cutoff)))

;; Compose filters
(->> (fs/glob "src" "**/*")
     (filter clojure-source?)
     (filter (partial recent? 7))
     (map str))

Atomic File Operations

;; Write to temp file, then move (atomic on most filesystems)
(let [target (fs/path "important-data.edn")
      tmp-file (fs/create-temp-file {:prefix ".tmp-"
                                       :suffix ".edn"
                                       :dir (fs/parent target)})]
  (try
    (spit tmp-file (pr-str data))
    (fs/move tmp-file target {:replace-existing true})
    (catch Exception e
      (fs/delete-if-exists tmp-file)
      (throw e))))

Common Use Cases and Recipes

Build Tool Tasks

;; Clean target directory
(defn clean []
  (when (fs/exists? "target")
    (fs/delete-tree "target")))

;; Copy resources
(defn copy-resources []
  (fs/create-dirs "target/resources")
  (fs/copy-tree "resources" "target/resources"))

;; Find all source files
(defn source-files []
  (fs/glob "src" "**/*.clj"))

File Backup

(defn backup-file [path]
  (let [backup-name (str path ".backup."
                         (System/currentTimeMillis))]
    (fs/copy path backup-name)))

(defn backup-directory [dir dest]
  (let [timestamp (System/currentTimeMillis)
        backup-dir (fs/path dest (str (fs/file-name dir)
                                      "-" timestamp))]
    (fs/copy-tree dir backup-dir)))

Log Rotation

(defn rotate-logs [log-dir max-age-days]
  (let [cutoff (- (System/currentTimeMillis)
                  (* max-age-days 24 60 60 1000))]
    (->> (fs/glob log-dir "*.log")
         (filter #(< (fs/file-time->millis
                      (fs/last-modified-time %))
                     cutoff))
         (run! fs/delete))))

File Synchronization

(defn sync-newer-files [src dest]
  (doseq [src-file (fs/glob src "**/*")
          :when (fs/regular-file? src-file)]
    (let [rel-path (fs/relativize src src-file)
          dest-file (fs/path dest rel-path)]
      (when (or (not (fs/exists? dest-file))
                (> (fs/file-time->millis (fs/last-modified-time src-file))
                   (fs/file-time->millis (fs/last-modified-time dest-file))))
        (fs/create-dirs (fs/parent dest-file))
        (fs/copy src-file dest-file {:replace-existing true})
        (println "Synced:" src-file)))))

Finding Duplicate Files

(require '[clojure.java.io :as io])
(import '[java.security MessageDigest])

(defn file-hash [path]
  (with-open [is (io/input-stream (fs/file path))]
    (let [digest (MessageDigest/getInstance "MD5")
          buffer (byte-array 8192)]
      (loop []
        (let [n (.read is buffer)]
          (when (pos? n)
            (.update digest buffer 0 n)
            (recur))))
      (format "%032x" (BigInteger. 1 (.digest digest))))))

(defn find-duplicates [dir]
  (->> (fs/glob dir "**/*")
       (filter fs/regular-file?)
       (group-by file-hash)
       (filter #(> (count (val %)) 1))
       (map (fn [[hash paths]]
              {:hash hash
               :size (fs/size (first paths))
               :files (map str paths)}))))

Error Handling and Edge Cases

;; Handle missing files gracefully
(when (fs/exists? "config.edn")
  (process-config (slurp "config.edn")))

;; Or with try-catch
(try
  (process-file "data.txt")
  (catch java.nio.file.NoSuchFileException e
    (println "File not found:" (.getMessage e)))
  (catch java.nio.file.AccessDeniedException e
    (println "Access denied:" (.getMessage e))))

;; Check permissions before operations
(when (and (fs/exists? "file.txt")
           (fs/readable? "file.txt"))
  (slurp "file.txt"))

;; Handle walk errors
(fs/walk-file-tree "."
  {:visit-file-failed (fn [path ex]
                        (println "Cannot access:" path)
                        :continue)})  ; Continue despite errors

Performance Tips

Use directory-stream for large directories: It's lazy and doesn't load all entries into memory
Filter early: Apply filters in glob patterns when possible rather than filtering in Clojure
Avoid repeated file system calls: Cache results like file-exists? checks
Use walk-file-tree for deep recursion: More efficient than recursive list-dir
Batch operations: Group multiple files when possible instead of individual operations

Testing and Mocking

;; Use with-temp-dir for tests
(deftest test-file-processing
  (fs/with-temp-dir [tmp-dir {}]
    (let [test-file (fs/path tmp-dir "test.txt")]
      (spit test-file "test data")
      (is (fs/exists? test-file))
      (is (= "test data" (slurp test-file)))
      ;; No cleanup needed - automatic
      )))

Platform-Specific Considerations

Windows

Use fs/unixify to normalize paths for cross-platform code
Hidden files require the hidden attribute, not just a leading dot
POSIX permission functions won't work

Unix/Linux/macOS

Full POSIX permissions support
XDG base directory functions available
Hidden files start with dot
Owner functions work

General

Always use fs/path to join paths - it handles separators correctly
Test on target platforms when possible
Use relative paths when portability matters

Integration with Babashka Tasks

;; In bb.edn
{:tasks
 {:requires ([babashka.fs :as fs])

  clean {:doc "Remove build artifacts"
         :task (fs/delete-tree "target")}

  test {:doc "Run tests"
        :task (do
                (doseq [test-file (fs/glob "test" "**/*_test.clj")]
                  (load-file (str test-file))))}

  build {:doc "Build project"
         :depends [clean]
         :task (do
                 (fs/create-dirs "target")
                 (println "Building..."))}}}

Quick Reference: Most Common Functions

;; Checking
(fs/exists? path)
(fs/directory? path)
(fs/regular-file? path)

;; Creating
(fs/create-dirs path)
(fs/create-file path)
(fs/create-temp-dir)

;; Reading/Writing
(slurp (fs/file path))
(spit (fs/file path) content)
(fs/read-all-lines path)
(fs/write-lines path lines)

;; Copying/Moving/Deleting
(fs/copy src dest)
(fs/copy-tree src dest)
(fs/move src dest)
(fs/delete path)
(fs/delete-tree path)

;; Finding
(fs/glob root "**/*.clj")
(fs/match root pattern {:recursive true})
(fs/list-dir dir)
(fs/which "executable")

;; Paths
(fs/path "dir" "file")
(fs/parent path)
(fs/file-name path)
(fs/extension path)
(fs/absolutize path)
(fs/relativize base target)

Additional Resources

Summary

The babashka.fs library provides a comprehensive, idiomatic Clojure interface for file system operations. Key strengths:

Cross-platform: Handles OS differences automatically
Composable: Functions work well together in pipelines
Efficient: Built on NIO.2 for good performance
Practical: Includes high-level functions for common tasks
Safe: Provides options for atomic operations and error handling

When writing file system code in Clojure or Babashka, reach for babashka.fs first - it's likely to have exactly what you need with a clean, functional API.