PERLIMPORTS(1p)

NAME
perlimports - A command line utility for cleaning up imports in your Perl code

VERSION
version 0.000058

SYNOPSIS
Create a config file called "perlimports.toml" at the top level of your application or repository:

    perlimports --create-config-file perlimports.toml

You can also have a config file with a leading ".".

    perlimports --create-config-file .perlimports.toml

For system-wide defaults, you can create a file in $XDG_HOME. Something like:

    perlimports --create-config-file ~/.config/perlimports/perlimports.toml

After you have set up the config file to your liking, you can do away with most command line switches, other than "-i", "--lint" or "--read-stdin".

Now, let's update a file in place. (Make sure you can revert the file if you need to, as "perlimports" will not make a backup.)

    perlimports -i foo.pl

Lint a file:

    perlimports --lint foo.pl

In-place edits on directories:

    perlimports -i lib t xt

Lint directories:

    perlimports --lint lib t xt

In-place edits on files and directories

    perlimports -i foo.pl lib t xt

Run "perlimports" on a file and only print the results to STDOUT.

    perlimports foo.pl

If some of your imported modules are in local directories, you can give some hints as to where to find them:

    perlimports -i --libs t/lib,/some/dir/lib foo.pl

Redirect output to a new file:

    perlimports foo.pl > foo.new.pl

DESCRIPTION
This distribution provides the "perlimports" command line interface (CLI), which automates the cleanup and maintenance of Perl "use" and "require" statements. Loosely inspired by goimports <https://pkg.go.dev/golang.org/x/tools/cmd/goimports>, this tool aims to be part of your linting and tidying workflow, in much the same way you might use perltidy or perlcritic.

For a detailed discussion of the problems this tool attempts to solve, see this "Conference in the Cloud" talk from June 2021: Where did that Symbol Come From? <https://www.youtube.com/watch?v=fKqxdTbGxYY>.

Slides for the above talk are also available:

     curl -O https://raw.githubusercontent.com/oalders/presentations/main/slides/6-perlimports/remark.html && open remark.html

MOTIVATION
Many Perl modules helpfully export functions and variables by default. These provide handy shortcuts when you're writing a quick or small script, but they can quickly become a maintenance burden as code grows organically. When code increases in complexity, it leads to greater costs in terms of development time. Conversely, reducing code complexity can speed up development. This tool aims to reduce complexity to further this goal.

While importing symbols by default or using export tags provides a convenient shorthand for getting work done, this shorthand requires the developer to retain knowledge of these defaults and tags in order to understand the code. "perlimports" aims to allow you to develop your code as you see fit, while still giving you a viable option of tidying your imports automatically. In much the same way as you might use perltidy to format your code, you can now automate the process of making your imports easier to understand. Let's look at some examples.

Where is this function defined?
    You may come across some code like this:

        use strict;
        use warnings;

        use HTTP::Request::Common;
        use LWP::UserAgent;

        my $ua = LWP::UserAgent->new;
        my $req = $ua->request( GET 'https://metacpan.org/' );
        print $req->content;

    Where does "GET" come from? If you're not familiar with HTTP::Request::Common, you may not realize that the statement "use HTTP::Request::Common" has implicitly imported the functions "GET", "HEAD", "PUT", "PATCH", "POST" and "OPTIONS" into to this block of code.

    What would happen if we used "perlimports" to import all needed functions explicitly? It might look something like this:

        use strict;
        use warnings;

        use HTTP::Request::Common qw( GET );
        use LWP::UserAgent ();

        my $ua = LWP::UserAgent->new;
        my $req = $ua->request( GET 'https://metacpan.org/' );
        print $req->content;

    The code above makes it immediately obvious where "GET" originates, which in turn makes it easier for us to look up its documentation. It has the added bonus of also not importing "HEAD", "PUT" or any of the other functions which HTTP::Request::Common exports by default. So, those functions cannot unwittingly be used later in the code. This makes for more understandable code for present day you, future you and any others tasked with reading your code at some future point.

    Keep in mind that this simple act can save much time for developers who are not intimately familiar with Perl and the default exports of many CPAN modules.

Are we even using all of these imports?
    Imagine the following import statement

        use HTTP::Status qw(
            is_cacheable_by_default
            is_client_error
            is_error
            is_info
            is_redirect
            is_server_error
            is_success
            status_message
        );

    followed by 3,000 lines of code. How do you know if all of these functions are actually being used? Were they ever used? You can grep all of these function names manually or you can remove them by trial and error to see what breaks. This is a doable solution, but it does not scale well to scripts and modules with many imports or to large code bases with many imports. Having an unmaintained list of imports is preferable to implicit imports, but it would be helpful to automate maintaining this list.

    perlimports can, in many situations, clean up your import statements and automate this maintenance burden away. This makes it easier for you to write clean code, which is easier to understand.

Are we even using all of these modules?
    In cases where code is implicitly importing from modules or where explicit imports are not being curated, it can be hard to discover which modules are no longer being used in a script, module or even a code base. Removing unused modules from code can lead to gains in performance and decrease in consumption of resources. Removing entire modules from your code base can decrease the number of dependencies which you need to manage and decrease friction in your your deployment process.

    "perlimports" can remove unused modules for you, making dependency management much easier.

Enforcing a consistent style
    Having a messy list of module imports makes your code harder to read. Imagine this:

        use Cpanel::JSON::XS;
        use Database::Migrator::Types qw( HashRef ArrayRef Object Str Bool Maybe CodeRef FileHandle RegexpRef );
        use List::AllUtils qw( uniq any );
        use LWP::UserAgent    q{};
        use Try::Tiny qw/ catch     try /;
        use WWW::Mechanize  q<>;

    perlimports turns the above list into:

        use Cpanel::JSON::XS ();
        use Database::Migrator::Types qw(
            ArrayRef
            Bool
            CodeRef
            FileHandle
            HashRef
            Maybe
            Object
            RegexpRef
            Str
        );
        use List::AllUtils qw( any uniq );
        use LWP::UserAgent ();
        use Try::Tiny qw( catch try);
        use WWW::Mechanize ();

    Where possible, perlimports will enforce a consistent style of parentheses and will also sort your imports and break up long lines. As mentioned above, if some imports are no longer in use, "perlimports" will helpfully remove these for you.

Import tags
    Import tags may obscure where symbols are coming from. While import tags provide a useful shorthand, they can contribute to code complexity by obscuring the origin of imported symbols. Consider:

        use HTTP::Status qw(:constants :is status_message);

    The above line imports the status_message() function as well *some other things* via ":constants" and ":is". What exactly are these things? We'll need to read the documentation to know for sure.

    "perlimports" can audit your code and expand the line above to list the symbols which you are actually importing. So, the line above might now look something like:

        use HTTP::Status qw(
            HTTP_ACCEPTED
            HTTP_BAD_REQUEST
            HTTP_CONTINUE
            HTTP_I_AM_A_TEAPOT
            HTTP_MOVED_PERMANENTLY
            HTTP_NO_CODE
            HTTP_NOT_FOUND
            HTTP_OK
            HTTP_PAYLOAD_TOO_LARGE
            HTTP_PERMANENT_REDIRECT
            HTTP_RANGE_NOT_SATISFIABLE
            HTTP_REQUEST_ENTITY_TOO_LARGE
            HTTP_REQUEST_RANGE_NOT_SATISFIABLE
            HTTP_REQUEST_URI_TOO_LARGE
            HTTP_TOO_EARLY
            HTTP_UNORDERED_COLLECTION
            HTTP_URI_TOO_LONG
            is_cacheable_by_default
            is_client_error
            is_error
            is_info
            is_redirect
            is_server_error
            is_success
            status_message
        );

    This is more verbose, but grepping your code will now reveal to you where something like "is_cacheable_by_default" gets defined. You have increased the lines of code, but you have also reduced complexity.

COMMAND LINE PARAMETERS

  --create-config-file
Expects a path to a ".toml" file which does not yet exist.

    --create-config-file perlimports.toml
    --create-config-file .perlimports.toml

For system-wide defaults, you can create a file in $XDG_HOME. Something like:

    perlimports --create-config-file ~/.config/perlimports/perlimports.toml

All of the above are default file locations which perlimports will search, unless "--config-file" has been provided as an argument.

  --config-file
Path to a perlimports config file. If this parameter is not supplied, we will look for a file called "perlimports.toml" or ".perlimports.toml" in the current directory and then look for a perlimports.toml in XDG_CONFIG_HOME (usually something like "$HOME/perlimports/perlimports.toml"). This behaviour can be disabled via "--no-config-file" as described below.

  --no-config-file
Prevents perlimports from automatically using a config file which is in one of the standard locations. Use this option if you want to define all of your behaviours on the command line.

  --filename|-f
The absolute or relative path to a file (or directory) to process.

    --filename path/to/file

    -f path/to/file

    -f path/to/dir

Note that if you do not provide a "--filename" we will fall back to checking @ARGV for any remaining args. So,

    perlimports --filename path/to/file

is equivalent to

    perlimports path/to/file

You may also pass multiple file and dir names.

    perlimports path/to/file path/to/other/file lib t xt

  --ignore-modules
A comma-separated list of module names which should be ignored by this script. Any modules in this list should remain unchanged after processing.

    --ignore-modules Foo,Foo::Bar

  --ignore-modules-filename
The absolute or relative path to a file which contains a lost of module names to ignore. (See above for behaviour). The pattern is one module name per line.

    Foo
    Foo::Bar

  --ignore-modules-pattern
A regular expression to match module names which should be ignored by this script. Any modules matched by this regular expression remain unchanged after processing.

    --ignore-modules-pattern '^(Foo|Foo::Bar)'

  --ignore-modules-pattern-filename
The absolute or relative path to a file which contains a list of regular expression that matches modules that should be ignored. (See above for behaviour). The pattern is one regular expression per line.

    ^Foo
    ^Foo::Bar

  --never-export-modules
A comma-separated list of module names which should never export symbols. If these modules are found, we will ensure that they have an empty import list. So, "use Foo;" becomes "use Foo ();".

    --never-export-modules Foo,Foo::Bar

  --never-export-modules-filename
The absolute or relative path to a file which contains a lost of module names which should never export symbols. (See above for behaviour). The pattern is one module name per line.

    Foo
    Foo::Bar

  --inplace-edit|-i
Edit the file in place rather than printing the result to STDOUT. Make sure you have a backup copy first.

    --inplace--edit
    -i

Edit the file in place rather than printing the result to STDOUT. Make sure you have a backup copy first.

  --json
(Experimental). If enabled, linting errors will be emitted as JSON objects with one object per error. This is intended to make it easier for editors to parse line numbers and column numbers as well as the error message. This flag can only be used in tandem with "--lint".

  --lint
If this argument is passed, "perlimports" will act as a linter, rather than a tidier. Failure (and success) messages will be reported on STDERR. Failures will also return a non-zero exit code.

This is still a bit experimental, so the output format and exit codes could change in a subsequent release.

This cannot be combined with tidying. So, passing "--lint --inplace-edit" will generate an error.

  --[no-]padding
"--padding" is enabled by default, so you only need to pass this arg if you want to be explicit. This setting adds whitespace inside the parentheses.

    # --padding
    use Foo qw( bar baz );

The "--no-padding" arg allows you to disable the additional padding inside parentheses.

    # --no-padding
    use Foo qw(bar baz);

  --[no-]tidy-whitespace
"--tidy-whitespace" is enabled by default. This means that use statements will be updated even when the only change is in whitespace. Disabling this can help reduce the churn involved when running "perlimports", especially if the codebase does not have automated tidying.

If you have changed from "--padding" to "--no-padding" or vice versa, you'll probably want to ensure that "--tidy-whitespace" has also been enabled so that you can see the whitespace changes.

  --libs
A comma separated list of module directories which are not in your @INC

    --libs lib,t/lib

  --[no-]preserve-duplicates
When enabled, only one use statement per module will be preserved. Defaults to preserving duplicate statements.

For example, when enabled the following text

    use Foo qw( bar );
    use Foo qw (baz );

will be converted to:

    use Foo qw( bar baz );

If left disabled, the above will probably be converted to:

    use Foo qw( bar baz );
    use Foo qw( bar baz );

This allows you to determine manually how you'd like to handle the imports in question. Use this setting with care.

  --[no-]preserve-unused
When enabled, unused modules will be removed. Defaults to preserving unused modules.

Enabling this may remove modules which are only present for the purposes of preloading or which aren't being detected for other reasons, so use this setting with care.

  --range-begin
Experimental. First line of a range to tidy or lint. The line ranges begin at 1 (not 0). This will select the appropriate range of lines from "STDIN". Requires "--read-stdin". Mostly useful for editors. Unless you're writing an editor plugin or extension, you can probably ignore this.

  --range-end
Experimental. Last line of a range to tidy or lint. The line ranges begin at 1 (not 0). This will select the appropriate range of lines from "STDIN". Requires "--read-stdin". Mostly useful for editors. Unless you're writing an editor plugin or extension, you can probably ignore this.

  --read-stdin
Read statements to process from STDIN rather than processing the entire file. This is intended for use by editors, like "vim". See the "INTEGRATIONS" section below for more information on how to set up an integration with your editor.

If this option is enabled, then "--inplace-edit|-i" is not available.

    --read-stdin

  --log-level|-l
Generally only useful for debugging. "notice" notifies about progress, like which file or snippet is currently being processed. "info" will generally log the errors which were swallowed as text was being processed. All levels are subject to change.

    --log-level notice
    --log-level info
    -l notice
    -l info

See <https://metacpan.org/pod/Log::Dispatch#LOG-LEVELS> for a list of available log levels. Log output defaults to STDERR. See "--log-filename" if you'd rather log to a file.

  --log-filename
Name of a file to redirect logs to, rather than STDERR.

  --help
Output a concise help menu, with a summary of available parameters.

    --help

  --verbose-help
Include the SYNOPSIS section from this page after printing the "--help" menu listed above.

ANNOTATIONS/IGNORING MODULES
Aside from the documented command line switches for ignoring modules, you can add annotations in your code.

    use Encode; ## no perlimports

The above will tell perlimports not to attempt a tidy of this line.

    ## no perlimports
    use Encode;
    use Cpanel::JSON::XS;
    ## use perlimports

    use POSIX ();

The above will tell perlimports not to tidy the two modules contained inside of the annotations.

Please note that since perlimports needs to know as much as possible about what's going on in a file, the annotations don't prevent modules from being loaded. It's only a directive to leave the lines in the file unchanged after processing.

INTEGRATIONS
You are encouraged to make this tool part of your automated tidying workflow. Some guidance on how to configure this follows.

  VIM
If you're a "vim" user, you can pipe your import statements to perlimports directly.

    :vnoremap <silent> im :!perlimports --read-stdin --filename '%:p'<CR>

The above statement will allow you to visually select one or more lines of code and have them updated in place by "perlimports". Once you have selected the code enter "im" to have your imports (re)formatted.

  VIM and ALE
If you use ALE with vim, you can add something like this to your "vim" configuration. Note that this function will save your buffer before running "perlimports".

    function! Perlimports(buffer) abort
      write
      return {
      \   'command': 'perlimports --read-stdin -f %s'
      \}
    endfunction

    let ale_fixers.perl = ['perlimports', 'perltidy']
    execute ale#fix#registry#Add('perlimports', 'Perlimports', ['perl'], 'Tidy Perl imports')

  Emacs and Flycheck
Flycheck <https://www.flycheck.org/> 35 and newer will suggest changes from "perlimports" automatically if it finds that it is installed.

  precious
If you're a <https://github.com/houseabsolute/precious> user, your configuration might look something like this:

    exclude = [
        # Used by Dist::Zilla
        ".build",
        "App-perlimports-*",
        "blib",
        "inc",
        "test-data",
        # All of these are generated by Dist::Zilla
        "t/00-*",
        "t/author-*",
        "t/release-*",
        "xt/author",
        "xt/release",
    ]

    [commands.perlimports]
    type = "both"
    include = [ "**/*.{pl,pm,t,psgi}" ]
    cmd = [ "perlimports" ]
    lint_flags = [ "--lint"]
    tidy_flags = [ "-i"]
    ok_exit_codes = 0
    expect_stderr = true

    [commands.perltidy]
    type = "both"
    include = [ "**/*.{pl,pm,t,psgi}" ]
    cmd = [ "perltidy", "--profile=$PRECIOUS_ROOT/perltidyrc" ]
    lint_flags = [ "--assert-tidy", "--no-standard-output", "--outfile=/dev/null" ]
    tidy_flags = [ "--backup-and-modify-in-place", "--backup-file-extension=/" ]
    ok_exit_codes = 0
    lint_failure_exit_codes = 2
    expect_stderr = true

Note that <https://github.com/houseabsolute/precious> runs plugins in order, so we've placed a perltidy config after perlimports. This is handy because perlimports could introduce changes which will later be reverted by perltidy. By running them sequentially we can avoid false positives which might be generated by perlimports changing an include which perltidy might revert.

For an up to date example, see the config file which this repository uses: <https://github.com/oalders/App-perlimports/blob/main/precious.toml>

  Code::TidyAll
If you're a Code::TidyAll user, you can configure "perlimports" as a GenericTransformer. Your configuration might look something like this:

    [GenericTransformer perlimports]
    select = **/*.{pl,pm,t,psgi}
    ignore = .build/**/*
    ignore = App-perlimports-*/**/*
    ignore = blib/**/*
    ignore = fatlib/**/*
    ignore = inc/**/*
    ignore = t/00-*
    ignore = t/author-*
    ignore = t/release-*
    ignore = t/zzz-*
    ignore = test-data/**/*
    ignore = xt/**/*
    ignore = xt/author/{pod-coverage,pod-spell,tidyall}.t
    argv = --libs lib,t/lib --no-preserve-duplicates --no-preserve-unused --log-filename /tmp/perlimports.txt --log-level debug
    cmd = perlimports
    file_flag = -f
    ok_exit_codes = 0
    weight = 1

Note that in this case we've set the lowest possible weight. This is because we want "perlimports" to run before any other plugin which may transform the file. For example, you'll want perltidy to run after "perlimports" to avoid having to re-tidy files after your use statements have been rewritten.

If you want to use "tidyall" to run just "perlimports" you'll need to do something like:

    tidyall --plugin "GenericTransformer perlimports" -a

For an up to date example, see the config file which this repository uses: <https://github.com/oalders/App-perlimports/blob/main/tidyall.ini>

RECIPES
Included are some examples to use if you want to employ your own file finding logic or prefer not to use config files. If this does not apply to you, you can safely skip this section.

Running perlimports on test files

    find t -type f |              \
      grep .t$ |                  \
      xargs perlimports           \
      --libs lib,t/lib            \
      --ignore-modules Test::More \
      --no-preserve-unused        \
      --no-preserve-duplicates    \
      --log-level debug           \
      -i

The above command:

•   finds all test files in "./t"

•   pipes them to "perlimports"

•   adds "lib" and "t/lib" to @INC

•   ignores the Test::More module

•   removes unused modules

•   removes duplicated use statements

•   displays debugging info

•   edits files in place ("-i")

Users of ack can make this a touch simpler:

      ack -f --perltest           \
      xargs perlimports           \
      --libs lib,t/lib            \
      --ignore-modules Test::More \
      --no-preserve-unused        \
      --no-preserve-duplicates    \
      --log-level debug           \
      -i

Running perlimports on modules:

    find lib -type f |         \
      grep .pm$ |              \
      xargs perlimports        \
      --libs lib               \
      --no-preserve-unused     \
      --no-preserve-duplicates \
      -i

The above command:

•   finds all .pm files in "./lib"

•   pipes them to "perlimports"

•   adds "lib" to @INC

•   removes unused modules

•   removes duplicated use statements

•   edits files in place ("-i")

We can also make this slightly shorter by using ack:

      ack -f --perl lib        \
      xargs perlimports        \
      --libs lib               \
      --no-preserve-unused     \
      --no-preserve-duplicates \
      -i

CAVEATS
There are lots of shenanigans that Perl modules can get up to. This code will not find exports for all of those cases, but it should only attempt to rewrite imports which it knows how to handle. Please file a bug report in all other cases.

SEE ALSO
Perl::Critic::Policy::TooMuchCode::ProhibitUnusedImport, Perl::Critic::Policy::TooMuchCode::ProhibitUnusedInclude and Perl::Critic::Policy::TooMuchCode::ProhibitUnusedConstant

AUTHOR
Olaf Alders <olaf@wundercounter.com>

COPYRIGHT AND LICENSE
This software is copyright (c) 2020 by Olaf Alders.

This is free software; you can redistribute it and/or modify it under the same terms as the Perl 5 programming language system itself.