org.eclipse.jgit.internal.storage.pack

## Class PackWriter

- java.lang.Object
- - org.eclipse.jgit.internal.storage.pack.PackWriter

    public class PackWriter
    extends Object

PackWriter class is responsible for generating pack files from specified set of objects from repository. This implementation produce pack files in format version 2.

Source of objects may be specified in two ways:

- (usually) by providing sets of interesting and uninteresting objects in repository - all interesting objects and their ancestors except uninteresting objects and their ancestors will be included in pack, or
- by providing iterator of `RevObject` specifying exact list and order of objects in pack

Typical usage consists of creating instance intended for some pack, configuring options, preparing the list of objects by calling `preparePack(Iterator)` or `preparePack(ProgressMonitor, Collection, Collection)`, and finally producing the stream with `writePack(ProgressMonitor, ProgressMonitor, OutputStream)`.

Class provide set of configurable options and `ProgressMonitor` support, as operations may take a long time for big repositories. Deltas searching algorithm is NOT IMPLEMENTED yet - this implementation relies only on deltas and objects reuse.

This class is not thread safe, it is intended to be used in one thread, with one instance per created pack. Subsequent calls to writePack result in undefined behavior.

### Nested Class Summary

Nested Classes
`static interface` `PackWriter.ObjectIdSet` A collection of object ids.
`static class` `PackWriter.PackingPhase` Possible states that a PackWriter can be in.
`class` `PackWriter.State` Summary of the current state of a PackWriter.
`static class` `PackWriter.Statistics` Summary of how PackWriter created the pack.

### Constructor Summary

Constructors
Constructor and Description
`PackWriter(ObjectReader reader)` Create a writer to load objects from the specified reader.
`PackWriter(PackConfig config, ObjectReader reader)` Create writer with a specified configuration.
`PackWriter(Repository repo)` Create writer for specified repository.
`PackWriter(Repository repo, ObjectReader reader)` Create writer for specified repository.

### Method Summary

All Methods Static Methods Instance Methods Concrete Methods Deprecated Methods
`void` `addObject(RevObject object)` Include one object to the output file.
`ObjectId` `computeName()` Computes SHA-1 of lexicographically sorted objects ids written in this pack, as used to name a pack file in repository.
`void` `excludeObjects(PackWriter.ObjectIdSet idx)` Add a pack index whose contents should be excluded from the result.
`ObjectToPack` `get(AnyObjectId id)` Lookup the ObjectToPack object for a given ObjectId.
`int` `getIndexVersion()` Returns the index format version that will be written.
`static Iterable<PackWriter>` `getInstances()`
`long` `getObjectCount()` Returns objects number in a pack file that was created by this writer.
`ObjectIdOwnerMap<ObjectIdOwnerMap.Entry>` `getObjectSet()` Returns the object ids in the pack file that was created by this writer.
`PackWriter.State` `getState()`
`PackWriter.Statistics` `getStatistics()`
`boolean` `isDeltaBaseAsOffset()` Check whether writer can store delta base as an offset (new style reducing pack size) or should store it as an object id (legacy style, compatible with old readers).
`boolean` `isIgnoreMissingUninteresting()`
`boolean` `isIndexDisabled()`
`boolean` `isReuseDeltaCommits()` Check if the writer will reuse commits that are already stored as deltas.
`boolean` `isReuseValidatingObjects()` Check if the writer validates objects before copying them.
`boolean` `isThin()`
`boolean` `isUseBitmaps()`
`boolean` `isUseCachedPacks()`
`boolean` `prepareBitmapIndex(ProgressMonitor pm)` Prepares the bitmaps to be written to the pack index.
`void` `preparePack(Iterator<RevObject> objectsSource)` Prepare the list of objects to be written to the pack stream.
`void` `preparePack(ProgressMonitor countingMonitor, Collection<? extends ObjectId> want, Collection<? extends ObjectId> have)` Deprecated. to be removed in 2.0; use the Set version of this method.
`void` `preparePack(ProgressMonitor countingMonitor, ObjectWalk walk, Collection<? extends ObjectId> interestingObjects, Collection<? extends ObjectId> uninterestingObjects)` Deprecated. to be removed in 2.0; use the Set version of this method.
`void` `preparePack(ProgressMonitor countingMonitor, ObjectWalk walk, Set<? extends ObjectId> interestingObjects, Set<? extends ObjectId> uninterestingObjects)` Prepare the list of objects to be written to the pack stream.
`void` `preparePack(ProgressMonitor countingMonitor, Set<? extends ObjectId> want, Set<? extends ObjectId> have)` Prepare the list of objects to be written to the pack stream.
`void` `release()` Release all resources used by this writer.
`void` `select(ObjectToPack otp, StoredObjectRepresentation next)` Select an object representation for this writer.
`void` `setDeltaBaseAsOffset(boolean deltaBaseAsOffset)` Set writer delta base format.
`void` `setIgnoreMissingUninteresting(boolean ignore)`
`void` `setIndexDisabled(boolean noIndex)`
`void` `setReuseDeltaCommits(boolean reuse)` Set the writer to reuse existing delta versions of commits.
`void` `setReuseValidatingObjects(boolean validate)` Enable (or disable) object validation during packing.
`void` `setShallowPack(int depth, Collection<? extends ObjectId> unshallow)` Configure this pack for a shallow clone.
`void` `setTagTargets(Set<ObjectId> objects)` Set the tag targets that should be hoisted earlier during packing.
`void` `setThin(boolean packthin)`
`void` `setUseBitmaps(boolean useBitmaps)`
`void` `setUseCachedPacks(boolean useCached)`
`boolean` `willInclude(AnyObjectId id)` Determine if the pack file will contain the requested object.
`void` `writeBitmapIndex(OutputStream bitmapIndexStream)` Create a bitmap index file to match the pack file just written.
`void` `writeIndex(OutputStream indexStream)` Create an index file to match the pack file just written.
`void` `writePack(ProgressMonitor compressMonitor, ProgressMonitor writeMonitor, OutputStream packStream)` Write the prepared pack to the supplied stream.

### Methods inherited from class java.lang.Object

`clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

### Constructor Detail

#### PackWriter

    public PackWriter(Repository repo)

Create writer for specified repository.

Objects for packing are specified in `preparePack(Iterator)` or `preparePack(ProgressMonitor, Collection, Collection)`.

Parameters:
  `repo` - repository where objects are stored.

#### PackWriter

    public PackWriter(ObjectReader reader)

Create a writer to load objects from the specified reader.

Objects for packing are specified in `preparePack(Iterator)` or `preparePack(ProgressMonitor, Collection, Collection)`.

Parameters:
  `reader` - reader to read from the repository with.

#### PackWriter

    public PackWriter(Repository repo,
                      ObjectReader reader)

Create writer for specified repository.

Objects for packing are specified in `preparePack(Iterator)` or `preparePack(ProgressMonitor, Collection, Collection)`.

Parameters:
  `repo` - repository where objects are stored.
  `reader` - reader to read from the repository with.

#### PackWriter

    public PackWriter(PackConfig config,
                      ObjectReader reader)

Create writer with a specified configuration.

Objects for packing are specified in `preparePack(Iterator)` or `preparePack(ProgressMonitor, Collection, Collection)`.

Parameters:
  `config` - configuration for the pack writer.
  `reader` - reader to read from the repository with.

### Method Detail

#### getInstances

    public static Iterable<PackWriter> getInstances()

Returns:
  all allocated, non-released PackWriters instances.

#### isDeltaBaseAsOffset

    public boolean isDeltaBaseAsOffset()

Check whether writer can store delta base as an offset (new style reducing pack size) or should store it as an object id (legacy style, compatible with old readers). Default setting:

Returns:
  true if delta base is stored as an offset; false if it is stored as an object id.

#### setDeltaBaseAsOffset

    public void setDeltaBaseAsOffset(boolean deltaBaseAsOffset)

Set writer delta base format. Delta base can be written as an offset in a pack file (new approach reducing file size) or as an object id (legacy approach, compatible with old readers). Default setting:

Parameters:
  `deltaBaseAsOffset` - boolean indicating whether delta base can be stored as an offset.

#### isReuseDeltaCommits

    public boolean isReuseDeltaCommits()

Check if the writer will reuse commits that are already stored as deltas.

Returns:
  true if the writer would reuse commits stored as deltas, assuming delta reuse is already enabled.

#### setReuseDeltaCommits

    public void setReuseDeltaCommits(boolean reuse)

Set the writer to reuse existing delta versions of commits.

Parameters:
  `reuse` - if true, the writer will reuse any commits stored as deltas. By default the writer does not reuse delta commits.

#### isReuseValidatingObjects

    public boolean isReuseValidatingObjects()

Check if the writer validates objects before copying them.

Returns:
  true if validation is enabled; false if the reader will handle object validation as a side-effect of it consuming the output.

#### setReuseValidatingObjects

    public void setReuseValidatingObjects(boolean validate)

Enable (or disable) object validation during packing.

Parameters:
  `validate` - if true the pack writer will validate an object before it is put into the output. This additional validation work may be necessary to avoid propagating corruption from one local pack file to another local pack file.

#### isThin

    public boolean isThin()

Returns:
  true if this writer is producing a thin pack.

#### setThin

    public void setThin(boolean packthin)

Parameters:
  `packthin` - a boolean indicating whether writer may pack objects with delta base object not within set of objects to pack, but belonging to party repository (uninteresting/boundary) as determined by set; this kind of pack is used only for transport; true - to produce thin pack, false - otherwise.

#### isUseCachedPacks

    public boolean isUseCachedPacks()

Returns:
  true to reuse cached packs. If true index creation isn't available.

#### setUseCachedPacks

    public void setUseCachedPacks(boolean useCached)

Parameters:
  `useCached` - if set to true and a cached pack is present, it will be appended onto the end of a thin-pack, reducing the amount of working set space and CPU used by PackWriter. Enabling this feature prevents PackWriter from creating an index for the newly created pack, so its only suitable for writing to a network client, where the client will make the index.

#### isUseBitmaps

    public boolean isUseBitmaps()

Returns:
  true to use bitmaps for ObjectWalks, if available.

#### setUseBitmaps

    public void setUseBitmaps(boolean useBitmaps)

Parameters:
  `useBitmaps` - if set to true, bitmaps will be used when preparing a pack.

#### isIndexDisabled

    public boolean isIndexDisabled()

Returns:
  true if the index file cannot be created by this PackWriter.

#### setIndexDisabled

    public void setIndexDisabled(boolean noIndex)

Parameters:
  `noIndex` - true to disable creation of the index file.

#### isIgnoreMissingUninteresting

    public boolean isIgnoreMissingUninteresting()

Returns:
  true to ignore objects that are uninteresting and also not found on local disk; false to throw a `MissingObjectException` out of `preparePack(ProgressMonitor, Collection, Collection)` if an uninteresting object is not in the source repository. By default, true, permitting gracefully ignoring of uninteresting objects.

#### setIgnoreMissingUninteresting

    public void setIgnoreMissingUninteresting(boolean ignore)

Parameters:
  `ignore` - true if writer should ignore non existing uninteresting objects during construction set of objects to pack; false otherwise - non existing uninteresting objects may cause `MissingObjectException`

#### setTagTargets

    public void setTagTargets(Set<ObjectId> objects)

Set the tag targets that should be hoisted earlier during packing.

Callers may put objects into this set before invoking any of the preparePack methods to influence where an annotated tag's target is stored within the resulting pack. Typically these will be clustered together, and hoisted earlier in the file even if they are ancient revisions, allowing readers to find tag targets with better locality.

Parameters:
  `objects` - objects that annotated tags point at.

#### setShallowPack

    public void setShallowPack(int depth,
                               Collection<? extends ObjectId> unshallow)

Configure this pack for a shallow clone.

Parameters:
  `depth` - maximum depth to traverse the commit graph
  `unshallow` - objects which used to be shallow on the client, but are being extended as part of this fetch

#### getObjectCount

    public long getObjectCount()
                        throws IOException

Returns objects number in a pack file that was created by this writer.

Returns:
  number of objects in pack.
Throws:
  `IOException` - a cached pack cannot supply its object count.

#### getObjectSet

    public ObjectIdOwnerMap<ObjectIdOwnerMap.Entry> getObjectSet()
                                                          throws IOException

Returns the object ids in the pack file that was created by this writer. This method can only be invoked after `writePack(ProgressMonitor, ProgressMonitor, OutputStream)` has been invoked and completed successfully.

Returns:
  number of objects in pack.
Throws:
  `IOException` - a cached pack cannot supply its object ids.

#### excludeObjects

    public void excludeObjects(PackWriter.ObjectIdSet idx)

Add a pack index whose contents should be excluded from the result.

Parameters:
  `idx` - objects in this index will not be in the output pack.

#### preparePack

    public void preparePack(Iterator<RevObject> objectsSource)
                     throws IOException

Prepare the list of objects to be written to the pack stream.

Iterator exactly determines which objects are included in a pack and order they appear in pack (except that objects order by type is not needed at input). This order should conform general rules of ordering objects in git - by recency and path (type and delta-base first is internally secured) and responsibility for guaranteeing this order is on a caller side. Iterator must return each id of object to write exactly once.

Parameters:
  `objectsSource` - iterator of object to store in a pack; order of objects within each type is important, ordering by type is not needed; allowed types for objects are `Constants.OBJ_COMMIT`, `Constants.OBJ_TREE`, `Constants.OBJ_BLOB` and `Constants.OBJ_TAG`; objects returned by iterator may be later reused by caller as object id and type are internally copied in each iteration.
Throws:
  `IOException` - when some I/O problem occur during reading objects.

#### preparePack

    @Deprecated
    public void preparePack(ProgressMonitor countingMonitor,
                                        Collection<? extends ObjectId> want,
                                        Collection<? extends ObjectId> have)
                                 throws IOException

Deprecated. to be removed in 2.0; use the Set version of this method.
Prepare the list of objects to be written to the pack stream.

Basing on these 2 sets, another set of objects to put in a pack file is created: this set consists of all objects reachable (ancestors) from interesting objects, except uninteresting objects and their ancestors. This method uses class `ObjectWalk` extensively to find out that appropriate set of output objects and their optimal order in output pack. Order is consistent with general git in-pack rules: sort by object type, recency, path and delta-base first.

Parameters:
  `countingMonitor` - progress during object enumeration.
  `want` - collection of objects to be marked as interesting (start points of graph traversal).
  `have` - collection of objects to be marked as uninteresting (end points of graph traversal).
Throws:
  `IOException` - when some I/O problem occur during reading objects.

#### preparePack

    @Deprecated
    public void preparePack(ProgressMonitor countingMonitor,
                                        ObjectWalk walk,
                                        Collection<? extends ObjectId> interestingObjects,
                                        Collection<? extends ObjectId> uninterestingObjects)
                                 throws IOException

Deprecated. to be removed in 2.0; use the Set version of this method.
Prepare the list of objects to be written to the pack stream.

Basing on these 2 sets, another set of objects to put in a pack file is created: this set consists of all objects reachable (ancestors) from interesting objects, except uninteresting objects and their ancestors. This method uses class `ObjectWalk` extensively to find out that appropriate set of output objects and their optimal order in output pack. Order is consistent with general git in-pack rules: sort by object type, recency, path and delta-base first.

Parameters:
  `countingMonitor` - progress during object enumeration.
  `walk` - ObjectWalk to perform enumeration.
  `interestingObjects` - collection of objects to be marked as interesting (start points of graph traversal).
  `uninterestingObjects` - collection of objects to be marked as uninteresting (end points of graph traversal).
Throws:
  `IOException` - when some I/O problem occur during reading objects.

#### preparePack

    public void preparePack(ProgressMonitor countingMonitor,
                            Set<? extends ObjectId> want,
                            Set<? extends ObjectId> have)
                     throws IOException

Prepare the list of objects to be written to the pack stream.

Basing on these 2 sets, another set of objects to put in a pack file is created: this set consists of all objects reachable (ancestors) from interesting objects, except uninteresting objects and their ancestors. This method uses class `ObjectWalk` extensively to find out that appropriate set of output objects and their optimal order in output pack. Order is consistent with general git in-pack rules: sort by object type, recency, path and delta-base first.

Parameters:
  `countingMonitor` - progress during object enumeration.
  `want` - collection of objects to be marked as interesting (start points of graph traversal).
  `have` - collection of objects to be marked as uninteresting (end points of graph traversal).
Throws:
  `IOException` - when some I/O problem occur during reading objects.

#### preparePack

    public void preparePack(ProgressMonitor countingMonitor,
                            ObjectWalk walk,
                            Set<? extends ObjectId> interestingObjects,
                            Set<? extends ObjectId> uninterestingObjects)
                     throws IOException

Prepare the list of objects to be written to the pack stream.

Basing on these 2 sets, another set of objects to put in a pack file is created: this set consists of all objects reachable (ancestors) from interesting objects, except uninteresting objects and their ancestors. This method uses class `ObjectWalk` extensively to find out that appropriate set of output objects and their optimal order in output pack. Order is consistent with general git in-pack rules: sort by object type, recency, path and delta-base first.

Parameters:
  `countingMonitor` - progress during object enumeration.
  `walk` - ObjectWalk to perform enumeration.
  `interestingObjects` - collection of objects to be marked as interesting (start points of graph traversal).
  `uninterestingObjects` - collection of objects to be marked as uninteresting (end points of graph traversal).
Throws:
  `IOException` - when some I/O problem occur during reading objects.

#### willInclude

    public boolean willInclude(AnyObjectId id)
                        throws IOException

Determine if the pack file will contain the requested object.

Parameters:
  `id` - the object to test the existence of.
Returns:
  true if the object will appear in the output pack file.
Throws:
  `IOException` - a cached pack cannot be examined.

#### get

    public ObjectToPack get(AnyObjectId id)

Lookup the ObjectToPack object for a given ObjectId.

Parameters:
  `id` - the object to find in the pack.
Returns:
  the object we are packing, or null.

#### computeName

    public ObjectId computeName()

Computes SHA-1 of lexicographically sorted objects ids written in this pack, as used to name a pack file in repository.

Returns:
  ObjectId representing SHA-1 name of a pack that was created.

#### getIndexVersion

    public int getIndexVersion()

Returns the index format version that will be written.

This method can only be invoked after `writePack(ProgressMonitor, ProgressMonitor, OutputStream)` has been invoked and completed successfully.

Returns:
  the index format version.

#### writeIndex

    public void writeIndex(OutputStream indexStream)
                    throws IOException

Create an index file to match the pack file just written.

This method can only be invoked after `writePack(ProgressMonitor, ProgressMonitor, OutputStream)` has been invoked and completed successfully. Writing a corresponding index is an optional feature that not all pack users may require.

Parameters:
  `indexStream` - output for the index data. Caller is responsible for closing this stream.
Throws:
  `IOException` - the index data could not be written to the supplied stream.

#### writeBitmapIndex

    public void writeBitmapIndex(OutputStream bitmapIndexStream)
                          throws IOException

Create a bitmap index file to match the pack file just written.

This method can only be invoked after `prepareBitmapIndex(ProgressMonitor)` has been invoked and completed successfully. Writing a corresponding bitmap index is an optional feature that not all pack users may require.

Parameters:
  `bitmapIndexStream` - output for the bitmap index data. Caller is responsible for closing this stream.
Throws:
  `IOException` - the index data could not be written to the supplied stream.

#### writePack

    public void writePack(ProgressMonitor compressMonitor,
                          ProgressMonitor writeMonitor,
                          OutputStream packStream)
                   throws IOException

Write the prepared pack to the supplied stream.

At first, this method collects and sorts objects to pack, then deltas search is performed if set up accordingly, finally pack stream is written.

All reused objects data checksum (Adler32/CRC32) is computed and validated against existing checksum.

Parameters:
  `compressMonitor` - progress monitor to report object compression work.
  `writeMonitor` - progress monitor to report the number of objects written.
  `packStream` - output stream of pack data. The stream should be buffered by the caller. The caller is responsible for closing the stream.
Throws:
  `IOException` - an error occurred reading a local object's data to include in the pack, or writing compressed object data to the output stream.

#### getStatistics

    public PackWriter.Statistics getStatistics()

Returns:
  description of what this PackWriter did in order to create the final pack stream. The object is only available to callers after `writePack(ProgressMonitor, ProgressMonitor, OutputStream)`

#### getState

    public PackWriter.State getState()

Returns:
  snapshot of the current state of this PackWriter.

#### release

    public void release()

Release all resources used by this writer.

#### addObject

    public void addObject(RevObject object)
                   throws IncorrectObjectTypeException

Include one object to the output file.

Objects are written in the order they are added. If the same object is added twice, it may be written twice, creating a larger than necessary file.

Parameters:
  `object` - the object to add.
Throws:
  `IncorrectObjectTypeException` - the object is an unsupported type.

#### select

    public void select(ObjectToPack otp,
                       StoredObjectRepresentation next)

Select an object representation for this writer.

An `ObjectReader` implementation should invoke this method once for each representation available for an object, to allow the writer to find the most suitable one for the output.

Parameters:
  `otp` - the object being packed.
  `next` - the next available representation from the repository.

#### prepareBitmapIndex

    public boolean prepareBitmapIndex(ProgressMonitor pm)
                               throws IOException

Prepares the bitmaps to be written to the pack index. Bitmaps can be used to speed up fetches and clones by storing the entire object graph at selected commits. This method can only be invoked after `writePack(ProgressMonitor, ProgressMonitor, OutputStream)` has been invoked and completed successfully. Writing a corresponding bitmap index is an optional feature that not all pack users may require.

Parameters:
  `pm` - progress monitor to report bitmap building work.
Returns:
  whether a bitmap index may be written.
Throws:
  `IOException` - when some I/O problem occur during reading objects.