Announcing linesieve 1.0: an unholy blend of grep, sed, awk, and Python, born out of spite

April 2023 ∙ three minute read ∙

Java is notoriously verbose, especially when used in a serious Enterprise Project™. naturally, I made linesieve, a Python text munging tool to split output into sections and match/sub/split with the full power of Python's re module.

Here's an example of using it on a file listing (delay added to make it look cool):

ls -1 /* | linesieve -s '.*:' show bin match ^d head -n2 output
ls -1 /* | linesieve -s '.*:' show bin match ^d head -n2 output

You can find short examples in the reference, and an advanced example below.

Features #

linesieve allows you to:

  • split text input into sections
  • apply filters to specific sections
  • search and highlight success/failure markers
  • match/sub/split with the full power of Python's re module
  • shorten paths, links and module names
  • chain filters into pipelines
  • colors!

Here's a list of filters available in 1.0:

  • head – Output the first part of sections.
  • tail – Output the last part of sections.
  • span – Output matching line spans.
  • match – Search for pattern (think grep).
  • split – Output selected parts of lines (think cut).
  • sub – Replace pattern (think sed s///g).
  • sub-paths – Shorten paths of existing files.
  • sub-cwd – Make working directory paths relative.
  • sub-link – Replace symlink targets.

Installing #

Install it using pip:

$ pip install --upgrade linesieve

You can find:

Example #

Here's a juicy Java example – this linesieve command:

linesieve \
span -v -X \
    --start '^ (\s+) at \s ( org\.junit\. | \S+ \. reflect\.\S+\.invoke )' \
    --end '^ (?! \s+ at \s )' \
    --repl '\1...' \
match -v '^\s+at \S+\.(rethrowAs|translateTo)IOException' \
sub-paths --include '{src,tst}/**/*.java' --modules-skip 1 \
sub -X '^( \s+ at \s+ (?! .+ \.\. | com\.example\. ) .*? ) \( .*' '\1' \
sub -X '^( \s+ at \s+ com\.example\. .*? ) \ ~\[ .*' '\1' \
sub -X '
    (?P<pre> \s+ at \s .*)
    (?P<cls> \w+ )
    (?P<mid> .* \( )
    (?P=cls) \.java
    (?P<suf> : .* )
    ' \

... shortens this 76 line traceback:

12:34:56.789 [main] ERROR com.example.someproject.somepackage.ThingDoer - exception while notifying done listener
java.lang.RuntimeException: listener failed
    at com.example.someproject.somepackage.ThingDoerTest$DummyListener.onThingDone( ~[tests/:?]
    at com.example.someproject.somepackage.ThingDoer.doThing( ~[library/:?]
    at com.example.otherproject.Framework.doAllTheThings( ~[example-otherproject-2.0.jar:2.0]
    at com.example.someproject.somepackage.ThingDoerTest.listenerException( ~[tests/:?]
    at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:?]
    at jdk.internal.reflect.NativeMethodAccessorImpl.invoke( ~[?:?]
    ... 60+ more lines of JUnit stuff we don't really care about ...
12:34:56.999 [main] INFO done

... to just 7 lines:

12:34:56.789 [main] ERROR ..ThingDoer - exception while notifying done listener
java.lang.RuntimeException: listener failed
    at ..ThingDoerTest$DummyListener.onThingDone(:420) ~[tests/:?]
    at ..ThingDoer.doThing(:69) ~[library/:?]
    at com.example.otherproject.Framework.doAllTheThings(:1066)
    at ..ThingDoerTest.listenerException(:666) ~[tests/:?]
12:34:56.999 [main] INFO done

Let's break that down a bit:

  • span gets rid of all the traceback lines coming from JUnit.
  • match -v skips some usually useless lines from stack traces.
  • sub-paths shortens and highlights the names of classes in the current project; com.example.someproject.somepackage.ThingDoer becomes ..ThingDoer (presumably that's enough info to open the file in your IDE).
  • The first sub gets rid of line numbers and JAR names for everything that's not either in the current project or in another com.example. package.
  • The second sub gets rid of JAR names for things in other com.example. packages.
  • The third sub gets rid of the source file name; ..ThingDoer.doThing( becomes ..ThingDoer.doThing(:69) (in Java, the file name matches the class name).

For an even more advanced example, see how to clean up Apache Ant output.

Anyway, that's it for now.

Learned something new today? Share this with others, it really helps!