git-annex/ walkthrough/ renaming files

    $ cd ~/annex
    $ git mv big_file my_cool_big_file
    $ mkdir iso
    $ git mv debian.iso iso/
    $ git commit -m moved

You can use any normal git operations to move files around, or even make copies or delete them.

Notice that, since annexed files are represented by symlinks, the symlink will break when the file is moved into a subdirectory. But, git-annex will fix this up for you when you commit -- it has a pre-commit hook that watches for and corrects broken symlinks.

Bitten again by "E" backend, wish "e" backend.

Changing file extension is arguably not a very common use case. But changing extension case is more common, especially when copying files from FAT filesystems through different ways (Card reader, plugging device directly), and causes troubles.

# Concrete case

A set of files were imported with extension NEF and JPG, by mistake (forgot to rename them first), using default (or is it?) backend SHA256E.

When using `git annex reinject --known` to cleanup another copy, files were not detected as known.

This will happen again and feels not very good.

# Workaround

Thanks to CandyAngel I know I can rename then use `git annex migrate --force` and `reinject` again.

# Better solution

A SHA256e backend (that squashes extensions to lowercase) would not have this problem.

# Additional information

Why not switch to plain SHA256 backend? I'm a little paranoid (which makes me a good fit for using git-annex overall).

One of the reason I use git-annex is that its storage format is actually simple, and files can be accessed even on a computer that can read the plain filesystem, without git or git-annex required. But in the lifetime of a filesystem, many bad things can happen and details of storage format may make a difference in case of major recovery (or even sanity check) scenario. Hashes without extension feel somehow uncomfortable. Lowercase extension backends feel much better.

Comment by https://launchpad.net/~stephane-gourichon-lpad — Tue Jul 22 18:11:47 2025
Nevermind - broken symlinks will be fixed on commit

Didn't catch this from the docs above:

"Notice that, since annexed files are represented by symlinks, the symlink will break when the file is moved into a subdirectory. But, git-annex will fix this up for you when you commit -- it has a pre-commit hook that watches for and corrects broken symlinks."

Comment by dusty — Tue Jul 22 18:11:47 2025
moving files with git mv can yield broken symlink

Hey Joey,

Just wondering how we rename/move files around into different directories within a repository without breaking the symbolic links. If we move to a different directory and the depth is different than where the files were originally in the repo then the symbolic link is broken.

Thanks

Comment by dusty — Tue Jul 22 18:11:47 2025
comment 3

This is basically the same as thinking that git should change the SHA1 used to refer to a file in the repository just because it was renamed. That does not normally make sense.

Including the original extension in the git-annex key is done to avoid confusing some poorly-behaved programs that expect to follow the symlink and find a file with an extension. It's best to ignore the extension part of the key, or if you like, you can change the backend from SHA256E to SHA256, and then new keys won't include that extension.

Comment by joey — Tue Jul 22 18:11:47 2025
comment 2

migrate takes the --force option (introduced 5.20150327), allowing you to migrate files to the backend they are already using, but using the new extension. It was actually added for this use case!

This may still incur a hash of the file though as, I believe, migrate will check the file at the old key is valid. You will also end up with 2 versions of the file in .git/annex/objects though (you can clean it up with unused).

Comment by CandyAngel — Tue Jul 22 18:11:47 2025
renaming oesn't work as expected when changing the extension

I tried to rename a `.gif` file to `.jpg`:

    $ git annex version
    git-annex version: 5.20140127
    build flags: S3 DBus Feeds Quvi TDFA CryptoHash
    key/value backends: SHA256E SHA1E SHA512E SHA224E SHA384E SKEIN256E SKEIN512E SHA256 SHA1 SHA512 SHA224 SHA384 SKEIN256 SKEIN512 WORM URL
    remote types: git gcrypt S3 bup directory rsync web tahoe glacier hook external
    local repository version: 5
    supported repository version: 5
    upgrade supported from repository versions: 0 1 2 4
    $ find .git/annex/objects -type f
    .git/annex/objects/JW/wq/SHA256E-s231374848--01ca5e0bb09d068f15ccbf9b064b72f573198bd7516ff8edc72e455a09bf9bd4.gif/SHA256E-s231374848--01ca5e0bb09d068f15ccbf9b064b72f573198bd7516ff8edc72e455a09bf9bd4.gif
    $ git mv foo.gif foo.jpg
    $ git commit -a -m renamed
    ...
    $ git annex fsck
    ...
    $ find .git/annex/objects -type f
    .git/annex/objects/JW/wq/SHA256E-s231374848--01ca5e0bb09d068f15ccbf9b064b72f573198bd7516ff8edc72e455a09bf9bd4.gif/SHA256E-s231374848--01ca5e0bb09d068f15ccbf9b064b72f573198bd7516ff8edc72e455a09bf9bd4.gif
    $ ls -l foo.jpg
    lrwxrwxrwx 1 pts pts 202 Oct 15 10:36 foo.jpg -> .git/annex/objects/JW/wq/SHA256E-s231374848--01ca5e0bb09d068f15ccbf9b064b72f573198bd7516ff8edc72e455a09bf9bd4.gif/SHA256E-s231374848--01ca5e0bb09d068f15ccbf9b064b72f573198bd7516ff8edc72e455a09bf9bd4.gif

I would expect that the last extension printed is .jpg, rather than .gif. How do I make git-annex fix that, preferably without calling `git annex fsck`, which is very slow, because it rehashes the file?

Comment by ptspts — Tue Jul 22 18:11:47 2025
Comments on this page are closed.