Determining the gem's list of files for the specification

Question

I've always used git to determine which files should go into the gem package:

gem.files = `git ls-files`.split "\n"

Unfortunately, this approach has recently proved to be inappropriate. I need a self-contained, pure-Ruby solution.

My first idea was to simply glob the entire directory, but that alone is likely to include unwanted files. So, after researching the problem, I came up with this:

# example.gemspec

directory = File.dirname File.expand_path __FILE__
dotfiles = %w(.gitignore .rvmrc)
ignore_file = '.gitignore'
file_list = []

Dir.chdir directory do
  ignored = File.readlines(ignore_file).map(&:chomp).reject { |glob| glob =~ /\A(#|\s*\z)/ }
  file_list.replace Dir['**/**'] + dotfiles
  file_list.delete_if do |file|
    File.directory?(file) or ignored.any? { |glob| File.fnmatch? glob, file }
  end
end

# Later...

gem.files = file_list

That seems a bit complex for a gemspec. It also does not fully support gitignore's pattern format. It currently seems to work but I'd rather not run into problems later.

Is there a simpler but robust way to compute the gem's list of files? Most gems apparently use git ls-files, and the ones that don't either use a solution similar to mine or specify the files manually.

Answer 1

With Rake

The easiest solution depending on rake to list all files from a directory, but exclude everything in the .gitignore file:

require 'rake/file_list'
Rake::FileList['**/*'].exclude(*File.read('.gitignore').split)

RubyGems

Official rubygems solution, list and exclude manually:

require 'rake'
spec.files = FileList['lib/*.rb',
                      'bin/*',
                      '[A-Z]*',
                      'test/*'].to_a

# or without Rake...
spec.files = Dir['lib/*.rb'] + Dir['bin/*']
spec.files += Dir['[A-Z]*'] + Dir['test/**/*']
spec.files.reject! { |fn| fn.include? "CVS" }

Bundler

Bundler solution, list manually:

s.files = Dir.glob("{lib,exe}/**/*", File::FNM_DOTMATCH).reject {|f| File.directory?(f) }

Note: rejecting directories is useless as gem will ignore them by default.

Vagrant

Vagrant solution to mimic git ls-files and taking care of .gitignore in pure ruby:

  # The following block of code determines the files that should be included
  # in the gem. It does this by reading all the files in the directory where
  # this gemspec is, and parsing out the ignored files from the gitignore.
  # Note that the entire gitignore(5) syntax is not supported, specifically
  # the "!" syntax, but it should mostly work correctly.
  root_path      = File.dirname(__FILE__)
  all_files      = Dir.chdir(root_path) { Dir.glob("**/{*,.*}") }
  all_files.reject! { |file| [".", ".."].include?(File.basename(file)) }
  all_files.reject! { |file| file.start_with?("website/") }
  all_files.reject! { |file| file.start_with?("test/") }
  gitignore_path = File.join(root_path, ".gitignore")
  gitignore      = File.readlines(gitignore_path)
  gitignore.map!    { |line| line.chomp.strip }
  gitignore.reject! { |line| line.empty? || line =~ /^(#|!)/ }

  unignored_files = all_files.reject do |file|
    # Ignore any directories, the gemspec only cares about files
    next true if File.directory?(file)

    # Ignore any paths that match anything in the gitignore. We do
    # two tests here:
    #
    #   - First, test to see if the entire path matches the gitignore.
    #   - Second, match if the basename does, this makes it so that things
    #     like '.DS_Store' will match sub-directories too (same behavior
    #     as git).
    #
    gitignore.any? do |ignore|
      File.fnmatch(ignore, file, File::FNM_PATHNAME) ||
        File.fnmatch(ignore, File.basename(file), File::FNM_PATHNAME)
    end
  end

Pathspec

Using pathspec gem Match Path Specifications, such as .gitignore, in Ruby!

See https://github.com/highb/pathspec-ruby

References

Ref: Bundler Vagrant RubyGems Rake easy solution

Answer 2

Hi,

You can list all files of your project with pure Ruby:

gem.files = Dir['**/*'].keep_if { |file| File.file?(file) }

Or you can do it manually, this solution is used by Ruby on Rails gems:

gem.files = Dir['lib/**/*'] + %w(.yardopts Gemfile LICENSE README.md Rakefile my_gem.gemspec)