Monday, August 28, 2006

Matt's Ruby Log Splitter

Matt Secoske posted a Log Splitter on his blog. I thought it was a fun exercise, so I wrote my own as well. My version is different in that I made the work of splitting files into a class of its own that understands the same protocol of IO stream (or the parts that I need right now). All you do is simply create it like any another File object, except add a block that evaluates when the file should be split into a new one. The code at the bottom uses it. I tried to keep close to the input parameters of Matt's original code. Later on, I inspired him and he came up with a DSL version. Remember, I whipped this code up pretty quickly and I was having a bit of fun with Ruby. Enjoy!
class SplitFile
def self.open(suffix, mode, &when_to_split)
self.new(suffix, mode, when_to_split)
end

def write_all(enumerable)
enumerable.each {|line| write(line) }
end

def write(line)
open_next() if should_split?(line) || @file.nil?
@file.write(line)
end

def close()
@file.close() unless @file.nil?
@file = nil
end

private
def initialize(suffix, mode, when_to_split)
@suffix = suffix
@mode = mode
@when_to_split = when_to_split
@file_count = -1
end

def open_next()
close()
file_name = next_file_name()
puts "Writing file #{file_name}"
@file = File.open(file_name, @mode)
end

def next_file_name()
@file_count += 1
"#{@file_count}_#{@suffix}"
end

def should_split?(line)
@when_to_split.call(line)
end
end


FILE_SUFFIX = ARGV[0] ? ARGV[0] : "split.log"
MAX_LINES = ARGV[1] ? ARGV[1].to_i : 500000

lines = 0
split=SplitFile.open(FILE_SUFFIX, 'w') do |line|
lines += 1
result = lines >= MAX_LINES
lines = 0 if result
result
end

begin
split.write_all($stdin)
ensure
split.close
end

No comments: