Package picard.illumina

## Class IlluminaBasecallsToSam

- java.lang.Object
- - picard.cmdline.CommandLineProgram
  - - picard.illumina.IlluminaBasecallsToSam

-     @DocumentedFeature
      public class IlluminaBasecallsToSam
      extends CommandLineProgram

  IlluminaBasecallsToSam transforms a lane of Illumina data file formats (bcl, locs, clocs, qseqs, etc.) into SAM or BAM file format.

  In this application, barcode data is read from Illumina data file groups, each of which is associated with a tile. Each tile may contain data for any number of barcodes, and a single barcode's data may span multiple tiles. Once the barcode data is collected from files, each barcode's data is written to its own SAM/BAM. The barcode data must be written in order; this means that barcode data from each tile is sorted before it is written to file, and that if a barcode's data does span multiple tiles, data collected from each tile must be written in the order of the tiles themselves.

  This class employs a number of private subclasses to achieve this goal. The TileReadAggregator controls the flow of operation. It is fed a number of Tiles which it uses to spawn TileReaders. TileReaders are responsible for reading Illumina data for their respective tiles from disk, and as they collect that data, it is fed back into the TileReadAggregator. When a TileReader completes a tile, it notifies the TileReadAggregator, which reviews what was read and conditionally queues its writing to disk, baring in mind the requirements of write-order described in the previous paragraph. As writes complete, the TileReadAggregator re-evaluates the state of reads/writes and may queue more writes. When all barcodes for all tiles have been written, the TileReadAggregator shuts down.

  The TileReadAggregator controls task execution using a specialized ThreadPoolExecutor. It accepts special Runnables of type PriorityRunnable which allow a priority to be assigned to the runnable. When the ThreadPoolExecutor is assigning threads, it gives priority to those PriorityRunnables with higher priority values. In this application, TileReaders are assigned lowest priority, and write tasks are assigned high priority. It is designed in this fashion to minimize the amount of time data must remain in memory (write the data as soon as possible, then discard it from memory) while maximizing CPU usage.

- - ### Field Summary

    Fields
    Modifier and Type | Field | Description
    --- | --- | ---
    `List<IlluminaUtil.IlluminaAdapterPair>` | `ADAPTERS_TO_CHECK` |
    `boolean` | `APPLY_EAMSS_FILTER` |
    `File` | `BARCODE_PARAMS` |
    `picard.illumina.ClusterDataToSamConverter.PopulateBarcode` | `BARCODE_POPULATION_STRATEGY` |
    `File` | `BARCODES_DIR` |
    `File` | `BASECALLS_DIR` |
    `Integer` | `FIRST_TILE` |
    `String` | `FIVE_PRIME_ADAPTER` |
    `Boolean` | `FORCE_GC` |
    `boolean` | `IGNORE_UNEXPECTED_BARCODES` |
    `boolean` | `INCLUDE_BARCODE_QUALITY` |
    `boolean` | `INCLUDE_BC_IN_RG_TAG` |
    `boolean` | `INCLUDE_NON_PF_READS` |
    `Integer` | `LANE` |
    `String` | `LIBRARY_NAME` |
    `File` | `LIBRARY_PARAMS` |
    `int` | `MAX_READS_IN_RAM_PER_TILE` |
    `int` | `MINIMUM_QUALITY` |
    `String` | `MOLECULAR_INDEX_BASE_QUALITY_TAG` |
    `String` | `MOLECULAR_INDEX_TAG` |
    `Integer` | `NUM_PROCESSORS` |
    `File` | `OUTPUT` |
    `String` | `PLATFORM` |
    `Integer` | `PROCESS_SINGLE_TILE` |
    `String` | `READ_GROUP_ID` |
    `String` | `READ_STRUCTURE` |
    `String` | `RUN_BARCODE` |
    `Date` | `RUN_START_DATE` |
    `String` | `SAMPLE_ALIAS` |
    `String` | `SEQUENCING_CENTER` |
    `List<String>` | `TAG_PER_MOLECULAR_INDEX` |
    `String` | `THREE_PRIME_ADAPTER` |
    `Integer` | `TILE_LIMIT` |
    `static String` | `USAGE` |

    - ### Fields inherited from class picard.cmdline.CommandLineProgram

      `COMPRESSION_LEVEL, CREATE_INDEX, CREATE_MD5_FILE, GA4GH_CLIENT_SECRETS, MAX_RECORDS_IN_RAM, QUIET, REFERENCE_SEQUENCE, referenceSequence, specialArgumentsCollection, TMP_DIR, USE_JDK_DEFLATER, USE_JDK_INFLATER, VALIDATION_STRINGENCY, VERBOSITY`

  - ### Constructor Summary

    Constructors
    Constructor | Description
    --- | ---
    `IlluminaBasecallsToSam()` |

  - ### Method Summary

    All Methods Static Methods Instance Methods Concrete Methods
    Modifier and Type | Method | Description
    --- | --- | ---
    `protected String[]` | `customCommandLineValidation()` | Put any custom command-line validation in an override of this method.
    `protected int` | `doWork()` | Do the work after command line has been parsed.
    `static void` | `main​(String[] args)` |

    - ### Methods inherited from class picard.cmdline.CommandLineProgram

      `getCommandLine, getCommandLineParser, getDefaultHeaders, getFaqLink, getMetricsFile, getStandardUsagePreamble, getStandardUsagePreamble, getVersion, hasWebDocumentation, instanceMain, instanceMainWithExit, makeReferenceArgumentCollection, parseArgs, requiresReference, setDefaultHeaders, useLegacyParser`

    - ### Methods inherited from class java.lang.Object

      `clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

- - ### Field Detail

    - #### USAGE

          public static final String USAGE

      See Also:
        Constant Field Values

    - #### BASECALLS_DIR

          @Argument(doc="The basecalls directory. ",
                    shortName="B")
          public File BASECALLS_DIR

    - #### BARCODES_DIR

          @Argument(doc="The barcodes directory with _barcode.txt files (generated by ExtractIlluminaBarcodes). If not set, use BASECALLS_DIR. ",
                    shortName="BCD",
                    optional=true)
          public File BARCODES_DIR

    - #### LANE

          @Argument(doc="Lane number. ",
                    shortName="L")
          public Integer LANE

    - #### OUTPUT

          @Argument(doc="Deprecated (use LIBRARY_PARAMS).  The output SAM or BAM file. Format is determined by extension.",
                    shortName="O",
                    mutex={"BARCODE_PARAMS","LIBRARY_PARAMS"})
          public File OUTPUT

    - #### RUN_BARCODE

          @Argument(doc="The barcode of the run.  Prefixed to read names.")
          public String RUN_BARCODE

    - #### SAMPLE_ALIAS

          @Argument(doc="Deprecated (use LIBRARY_PARAMS).  The name of the sequenced sample",
                    shortName="ALIAS",
                    mutex={"BARCODE_PARAMS","LIBRARY_PARAMS"})
          public String SAMPLE_ALIAS

    - #### READ_GROUP_ID

          @Argument(doc="ID used to link RG header record with RG tag in SAM record.  If these are unique in SAM files that get merged, merge performance is better.  If not specified, READ_GROUP_ID will be set to <first 5 chars of RUN_BARCODE>.<LANE> .",
                    shortName="RG",
                    optional=true)
          public String READ_GROUP_ID

    - #### LIBRARY_NAME

          @Argument(doc="Deprecated (use LIBRARY_PARAMS).  The name of the sequenced library",
                    shortName="LIB",
                    optional=true,
                    mutex={"BARCODE_PARAMS","LIBRARY_PARAMS"})
          public String LIBRARY_NAME

    - #### SEQUENCING_CENTER

          @Argument(doc="The name of the sequencing center that produced the reads.  Used to set the @RG->CN header tag.")
          public String SEQUENCING_CENTER

    - #### RUN_START_DATE

          @Argument(doc="The start date of the run.",
                    optional=true)
          public Date RUN_START_DATE

    - #### PLATFORM

          @Argument(doc="The name of the sequencing technology that produced the read.",
                    optional=true)
          public String PLATFORM

    - #### INCLUDE_BC_IN_RG_TAG

          @Argument(doc="Whether to include the barcode information in the @RG->BC header tag. Defaults to false until included in the SAM spec.")
          public boolean INCLUDE_BC_IN_RG_TAG

    - #### READ_STRUCTURE

          @Argument(doc="A description of the logical structure of clusters in an Illumina Run, i.e. a description of the structure IlluminaBasecallsToSam assumes the  data to be in. It should consist of integer/character pairs describing the number of cycles and the type of those cycles (B for Sample Barcode, M for molecular barcode, T for Template, and S for skip).  E.g. If the input data consists of 80 base clusters and we provide a read structure of \"28T8M8B8S28T\" then the sequence may be split up into four reads:\n* read one with 28 cycles (bases) of template\n* read two with 8 cycles (bases) of molecular barcode (ex. unique molecular barcode)\n* read three with 8 cycles (bases) of sample barcode\n* 8 cycles (bases) skipped.\n* read four with 28 cycles (bases) of template\nThe skipped cycles would NOT be included in an output SAM/BAM file or in read groups therein.",
                    shortName="RS")
          public String READ_STRUCTURE

    - #### BARCODE_PARAMS

          @Argument(doc="Deprecated (use LIBRARY_PARAMS).  Tab-separated file for creating all output BAMs for barcoded run with single IlluminaBasecallsToSam invocation.  Columns are BARCODE, OUTPUT, SAMPLE_ALIAS, and LIBRARY_NAME.  Row with BARCODE=N is used to specify a file for no barcode match",
                    mutex={"OUTPUT","SAMPLE_ALIAS","LIBRARY_NAME","LIBRARY_PARAMS"})
          public File BARCODE_PARAMS

    - #### LIBRARY_PARAMS

          @Argument(doc="Tab-separated file for creating all output BAMs for a lane with single IlluminaBasecallsToSam invocation.  The columns are OUTPUT, SAMPLE_ALIAS, and LIBRARY_NAME, BARCODE_1, BARCODE_2 ... BARCODE_X where X = number of barcodes per cluster (optional).  Row with BARCODE_1 set to \'N\' is used to specify a file for no barcode match.  You may also provide any 2 letter RG header attributes (excluding PU, CN, PL, and DT)  as columns in this file and the values for those columns will be inserted into the RG tag for the BAM file created for a given row.",
                    mutex={"OUTPUT","SAMPLE_ALIAS","LIBRARY_NAME","BARCODE_PARAMS"})
          public File LIBRARY_PARAMS

    - #### ADAPTERS_TO_CHECK

          @Argument(doc="Which adapters to look for in the read.")
          public List<IlluminaUtil.IlluminaAdapterPair> ADAPTERS_TO_CHECK

    - #### FIVE_PRIME_ADAPTER

          @Argument(doc="For specifying adapters other than standard Illumina",
                    optional=true)
          public String FIVE_PRIME_ADAPTER

    - #### THREE_PRIME_ADAPTER

          @Argument(doc="For specifying adapters other than standard Illumina",
                    optional=true)
          public String THREE_PRIME_ADAPTER

    - #### NUM_PROCESSORS

          @Argument(doc="The number of threads to run in parallel. If NUM_PROCESSORS = 0, number of cores is automatically set to the number of cores available on the machine. If NUM_PROCESSORS < 0, then the number of cores used will be the number available on the machine less NUM_PROCESSORS.")
          public Integer NUM_PROCESSORS

    - #### FIRST_TILE

          @Argument(doc="If set, this is the first tile to be processed (used for debugging).  Note that tiles are not processed in numerical order.",
                    mutex="PROCESS_SINGLE_TILE",
                    optional=true)
          public Integer FIRST_TILE

    - #### TILE_LIMIT

          @Argument(doc="If set, process no more than this many tiles (used for debugging).",
                    optional=true)
          public Integer TILE_LIMIT

    - #### PROCESS_SINGLE_TILE

          @Argument(doc="If set, process only the tile number given and append the tile number to the output file name.",
                    mutex="FIRST_TILE",
                    optional=true)
          public Integer PROCESS_SINGLE_TILE

    - #### FORCE_GC

          @Argument(doc="If true, call System.gc() periodically.  This is useful in cases in which the -Xmx value passed is larger than the available memory.")
          public Boolean FORCE_GC

    - #### APPLY_EAMSS_FILTER

          @Argument(doc="Apply EAMSS filtering to identify inappropriately quality scored bases towards the ends of reads and convert their quality scores to Q2.")
          public boolean APPLY_EAMSS_FILTER

    - #### MAX_READS_IN_RAM_PER_TILE

          @Argument(doc="Configure SortingCollections to store this many records before spilling to disk. For an indexed run, each SortingCollection gets this value/number of indices.")
          public int MAX_READS_IN_RAM_PER_TILE

    - #### MINIMUM_QUALITY

          @Argument(doc="The minimum quality (after transforming 0s to 1s) expected from reads.  If qualities are lower than this value, an error is thrown.The default of 2 is what the Illumina\'s spec describes as the minimum, but in practice the value has been observed lower.")
          public int MINIMUM_QUALITY

    - #### INCLUDE_NON_PF_READS

          @Argument(doc="Whether to include non-PF reads",
                    shortName="NONPF",
                    optional=true)
          public boolean INCLUDE_NON_PF_READS

    - #### IGNORE_UNEXPECTED_BARCODES

          @Argument(doc="Whether to ignore reads whose barcodes are not found in LIBRARY_PARAMS.  Useful when outputting BAMs for only a subset of the barcodes in a lane.",
                    shortName="IGNORE_UNEXPECTED")
          public boolean IGNORE_UNEXPECTED_BARCODES

    - #### MOLECULAR_INDEX_TAG

          @Argument(doc="The tag to use to store any molecular indexes.  If more than one molecular index is found, they will be concatenated and stored here.",
                    optional=true)
          public String MOLECULAR_INDEX_TAG

    - #### MOLECULAR_INDEX_BASE_QUALITY_TAG

          @Argument(doc="The tag to use to store any molecular index base qualities.  If more than one molecular index is found, their qualities will be concatenated and stored here (.i.e. the number of \"M\" operators in the READ_STRUCTURE)",
                    optional=true)
          public String MOLECULAR_INDEX_BASE_QUALITY_TAG

    - #### TAG_PER_MOLECULAR_INDEX

          @Argument(doc="The list of tags to store each molecular index.  The number of tags should match the number of molecular indexes.",
                    optional=true)
          public List<String> TAG_PER_MOLECULAR_INDEX

    - #### BARCODE_POPULATION_STRATEGY

          @Argument(doc="When should the sample barcode (as read by the sequencer) be placed on the reads in the BC tag?")
          public picard.illumina.ClusterDataToSamConverter.PopulateBarcode BARCODE_POPULATION_STRATEGY

    - #### INCLUDE_BARCODE_QUALITY

          @Argument(doc="Should the barcode quality be included when the sample barcode is included?")
          public boolean INCLUDE_BARCODE_QUALITY

  - ### Constructor Detail

    - #### IlluminaBasecallsToSam

          public IlluminaBasecallsToSam()

  - ### Method Detail

    - #### doWork

          protected int doWork()

      Description copied from class: `CommandLineProgram`
      Do the work after command line has been parsed. RuntimeException may be thrown by this method, and are reported appropriately.
      Specified by:
        `doWork` in class `CommandLineProgram`
      Returns:
        program exit status.

    - #### main

          public static void main​(String[] args)

    - #### customCommandLineValidation

          protected String[] customCommandLineValidation()

      Put any custom command-line validation in an override of this method. clp is initialized at this point and can be used to print usage and access args. Any options set by command-line parser can be validated.
      Overrides:
        `customCommandLineValidation` in class `CommandLineProgram`
      Returns:
        null if command line is valid. If command line is invalid, returns an array of error message to be written to the appropriate place.