iggy: ignore file processor for jvm

As a core team member for Swagger Codegen, one of the major features I’ve added in the last year was support for .swagger-codegen-ignore. This ignore file allows for defining more complex rules to prevent Swagger Codegen from overwriting existing files.

Previously, we only supported conditionally generating apis, models, their respective docs, and supporting files. You could list zero files (i.e. skip everything in that category) or list one or more filename in csv format. There were a few problems with this previous approach:

• Users didn’t know how to define the excludes.
  • Should they be absolute or relative paths?
  • Should they include file extensions?
  • Just the Swagger Definition’s schema name?
  • Should it be lowercase, title cased, or lower camel cased.
• It didn’t allow for excluding other files or folders unknown to Swagger Codegen at the time of the definition. This means that a generator in Swagger Codegen could create a new filename in a subsequent release, which may or may not be overwritten.
• There was no support for pattern-based exclusions

To account for these issues, I followed the manpage for .gitignore to design a simple package for excluding files. In less than 1000 LOC (including headers and comments), I created a solution that I was pretty happy with.

Since then, I’ve extracted the module to a project called iggy and published to Maven Central so other projects can benefit from the code. Check out the README, tests, and examples for full features.

iggy allows you to define the filename used for ignore. For example, the code included in Swagger Codegen supports only .swagger-codegen-ignore in the directory root of the generated code, but we had an issue filed about how to allow for complex exclusions on initial generation from the maven plugin. I submitted a fix that allows users to define an external ignore file to be consumed by the maven plugin, and this file is used as an override to the generated root’s ignore file if users generate over existing code.

It’s pretty nifty.

Usage

Given a directory structure:

.
├── level1
│   ├── a.txt
│   ├── level1file
│   └── level2
│       ├── b.txt
│       ├── level2file
│       └── level3
│           ├── c.txt
│           └── level3file
└── rootedfile

And the following ignore file:

# This is an example ignore file

# This matches only a.txt under level1 directory (relative to .ignore)
level1/a.txt

# This recursively matches: c.text
**/c.txt

# This recursively matches files ending in: file
**/*file

# This matches only against the directory in which .ignore exists, negating from the recursive *file pattern above
!/rootedfile

# This allows any file recursively named b with a three-character extension
!**/b.???

We can easily evaluate whether or not files in the hierarchy are marked for inclusion or exclusion.

This example is included in the repo under examples/example and is a kotlin script:

#!/bin/sh
exec kotlins -cp `mvncp us.jimschubert:iggy:1.0.0` "$0" "$@"
!#
// To run, see https://github.com/andrewoma/kotlin-script
// NOTE: This file will have apparent syntax issues in your IDE.
import us.jimschubert.iggy.IgnoreProcessor
import java.io.File

fun main(args: Array<String>) {
    val processor = IgnoreProcessor(".")
    val currentDirectory = File(".")

    fun walk(current: File) {
        current.listFiles().forEach { file ->
            if(file.isDirectory()) walk(file)
            else System.out.println("$file is allowed: ${processor.allowsFile(file)}")
        }
    }

    walk(currentDirectory)
}

Running this example gives the following output:

$ ./example
/Users/jim/bin/mvncp:267: warning: shadowing outer local variable - opts
./.ignore is allowed: true
./example is allowed: true
./level1/a.txt is allowed: false
./level1/level1file is allowed: false
./level1/level2/b.txt is allowed: true
./level1/level2/level2file is allowed: false
./level1/level2/level3/c.txt is allowed: false
./level1/level2/level3/level3file is allowed: false
./rootedfile is allowed: true

The API for IgnoreProcessor is pretty simple. First, you construct the class with the directory location of the ignore file (and optionally an ignore filename override). Then, you provide a file to IgnoreProcessor#allowsFile and it returns true or false based on your ignore file’s definitions.

You can also construct IgnoreProcessor with a File.

Notes

Unlike gitignore, iggy doesn’t traverse all directories to the defined hierarchy root. There is a single source of ignore definitions.

iggy uses java.nio.file.PathMatcher with conditionally defined glob patterns based on lines defined in your ignore file. There are some OS-specific things (with Windows, really) where case sensitive evaluation may be unintuitive.

Code

The official repository is on Gitlab at https://gitlab.com/jimschubert/iggy. There is a “mirror” on GitHub.