git-annex/ git-annex-unused

# NAME

git-annex unused - look for unused file content

# SYNOPSIS

git annex unused

# DESCRIPTION

Checks the annex for data that does not correspond to any files present in any tag or branch, or in the git index, and prints a numbered list of the data.

After running this command, you can use the `--unused` option with many other git-annex commands to operate on all the unused data that was found.

For example, to move all unused data to origin:

    git annex unused; git annex move --unused --to origin

# OPTIONS

- `--fast`

  Only show unused temp and bad files.

- `--from=repository`

  Check for unused data that is located in a repository.

  The repository should be specified using the name of a configured remote, or the UUID or description of a repository.

- `--used-refspec=+ref:-ref`

  By default, any data that the git index uses, or that any refs in the git repository point to is considered to be used. If you only want to use some refs, you can use this option to specify the ones to use. Data that is not in the specified refs (and not used by the index) will then be considered unused.

  See REFSPEC FORMAT below for details of the format of this setting.

  The git configuration annex.used-refspec can be used to configure this in a more permanent fashion.

- `--json`

  Enable JSON output. This is intended to be parsed by programs that use git-annex.

- `--json-error-messages`

  Messages that would normally be output to standard error are included in the JSON instead.

- Also the git-annex-common-options(1) can be used.

# REFSPEC FORMAT

The refspec format for --used-refspec and annex.used-refspec is a colon-separated list of additions and removals of refs. A somewhat contrived example:

    +refs/heads/*:+HEAD^:+refs/tags/*:-refs/tags/old-tag:reflog

This adds all refs/heads/ refs, as well as the previous version of HEAD. It also adds all tags, except for old-tag. And it adds all refs from the reflog.

The default behavior is equivilant to `--used-refspec=+refs/*:+HEAD`

The refspec is processed by starting with an empty set of refs, and walking the list in order from left to right.

Each + using a glob is matched against all relevant refs (a subset of `git show-ref`) and all matching refs are added to the set. For example, "+refs/remotes/*" adds all remote refs.

Each + without a glob adds the literal value to the set. For example, "+HEAD^" adds "HEAD^".

Each - is matched against the set of refs accumulated so far. Any refs with names that match are removed from the set.

"reflog" adds all the refs from the reflog. This will make past versions of files not be considered to be unused until the ref expires from the reflog (by default for 90 days). Note that this may make git-annex unused take some time to complete, it if needs to check every ref from the reflog.

# SEE ALSO

git-annex(1)

git-annex-dropunused(1)

git-annex-addunused(1)

git-annex-whereused(1)

git-annex-oldkeys(1)

# AUTHOR

Joey Hess id@joeyh.name

Warning: Automatically converted into a man page by mdwn2man. Edit with care.

comment 9
"I synced to all remotes and dropped everything in 'here'" -- git-annex-unused "Checks the annex" for the unused contents (unless `--from=repository` is used), so if you dropped everything in `here`, there's nothing to find. But it seems from `du` results that contents wasn't actually dropped? git-annex-whereis tells where git-annex thinks contents is.
Comment by Ilya_Shlyakhter — Tue Jul 22 18:11:47 2025
dropping contents of old keys after migration

"May I delete them" -- `git-annex-drop --force` may be safer, as it also updates location tracking. You might also want to git-annex-dead the dropped keys to prevent git-annex-fsck from complaining about lost contents.

Re: why git-annex-unused isn't finding the unused contents, try running it with `--used-refspec=+HEAD`, and make sure `annex.used-refspec` git config is not set. Note that this will mark as unused any annexed contents not referenced from the latest tree of the HEAD branch, e.g. annexed files that were removed in some older commit.

Comment by Ilya_Shlyakhter — Tue Jul 22 18:11:47 2025
comment 7
Did you commit after git-migrate? Does the worktree have any uncommitted changes?
Comment by Ilya_Shlyakhter — Tue Jul 22 18:11:47 2025
P.P.S. i dropped all local copies
I forgot to tell that after migrating I synced to all remotes and dropped everything in 'here', that's why I was expecting no more objects locally.
Comment by atrent — Tue Jul 22 18:11:47 2025
P.S. they are all SHA256E
the "lost" objects are all SHA256E (with the "E")
Comment by atrent — Tue Jul 22 18:11:47 2025
can't find unused objects

I recently migrated an annex to SHA256 (without "E") and I'm now trying to clean the repo from unused data. I have a strange situation: there are 62G of unused objects:

    $ du -ks .git/annex/objects/
    64334024    .git/annex/objects/

but 'git annex unused' gives me only:

    $ git annex unused
    unused ...
    Some annexed data is no longer used by any files:
    NUMBER  KEY
    1       SHA256E-s27--32efec98dc9e05442fc2385bb85d855a8c7824c68abd4ab5bf55a4dfe412b334.pdf
    (To see where data was previously used, try: git log --stat --no-textconv -S'KEY')
    To remove unwanted data: git-annex dropunused NUMBER
    ok

I've checked (through a small shell script) that none of the object is in fact referenced by any symlink...

May I delete them? Shall I do some other checking/fscking/repairing?

Thank you

Comment by atrent — Tue Jul 22 18:11:47 2025
comment 3

Well, this only finds old versions of files, it does not delete them.

Even if you choose to then run git-annex-dropunused, it won't lose data unless you force it to.

Comment by joey — Tue Jul 22 18:11:47 2025
comment 2
If I understand correctly, git-annex-unused will by default cause old versions of files to be lost?
Comment by Ilya_Shlyakhter — Tue Jul 22 18:11:47 2025
comment 1

@mark The `--from` option can be used with any remote, whether it's a normal git remote or a special remote.

Comment by joey — Tue Jul 22 18:11:47 2025
comment 12

@meribold there is not, but I've opened a todo, git-annex-unused --history

Comment by joey — Tue Jul 22 18:11:47 2025
comment 12
Is it possible to only list files that are not present in any branch/tag/ref and are also not present in any ancestor commit of any branch/tag/ref?
Comment by meribold — Tue Jul 22 18:11:47 2025
comment 11

Generally if git-annex unused does not identify some object as unused, that you expected would be unused, it's because it found a branch that still uses that object. It might be some old branch (or tag) that you made and forgot about, it might be a remote tracking branch, either for a remote that you forgot to sync the changes to, or for an old remote that got removed but in a way that left the tracking branch around. annex.used-refspec can also pull in other, non-branch refs.

Comment by joey — Tue Jul 22 18:11:47 2025
Special remotes
Does this work for special remotes? Or only remotes that are full repositories?
Comment by mark — Tue Jul 22 18:11:47 2025
Comments on this page are closed.