git-annex/ tips/ android sync with adb

While git-annex can be installed on your Android device, it might be easier not to install it there, but run it on your computer using `adb` to pull and push changes to the Android device.

A few reasons for going this route:

- Easier than installing git-annex on Android.
- Avoids needing to type commands into a terminal on Android.
- Avoids problems with putting a git-annex repository on Android's `/sdcard`, which is crippled by not supporting hard links etc.

All you should need is a USB cable (or adb over wifi), and the `adb` command.

## setting it up

First, initialize your git-annex repository on your computer, if you haven't already.

Then, in that repository, set up an adb special remote:

    git-annex initremote android type=adb androiddirectory=/sdcard encryption=none exporttree=yes importtree=yes

The above example syncs with the /sdcard directory of the Android device. That can be a lot of files, so you may want a more limited directory. See the sample workflows below for some more examples.

Next, configure how trees of files imported from it will be merged into your git repository.

        git config remote.android.annex-tracking-branch master:android

Setting "master:android" makes the phone be treated as containing a branch of the master branch, and makes all its files appear to be inside a subdirectory named `android`. If you want its files to not be in a subdirectory, set it to "master" instead.

Finally, you may want to configure a preferred content expression for the remote. That will limit both what is exported to it, and what is imported from it. If you want to fully sync all files, you don't need to do this.

For example, to limit the files that get imported and exported to sound files:

    git annex wanted android 'include=*.mp3 or include=*.ogg'

## syncing with it

    git annex sync --content android

This command does a bi-directional sync with the phone, first importing new and changed files from it, merging that into the master branch, and then exporting from the master branch back to the android device so any modifications you have made get synced over to it.

See git-annex-import, [[git-annex-export, and git-annex-sync for more details, and bear in mind that you can also use commands like these to only import from or export to the android device:

    git annex import master:android --from android
    git annex merge android/master
    git annex export master:android --to android

## sample workflows

### photos

Set up the remote to use the /sdcard/DCIM directory where the phone's camera stores them.

    git-annex initremote android type=adb androiddirectory=/sdcard/DCIM encryption=none exporttree=yes importtree=yes

The annex-tracking-branch can be the same as before, to limit the files that are synced to those in an android directory:

    git config remote.android.annex-tracking-branch master:android

If you don't want to keep old photos on your Android device, you can simply `git mv` the files from the android directory to another directory, and the next sync with the phone will delete them from the Android device:

    git mv android/* .
    git annex sync --content

### music and podcasts

You could set up the remote to use the /sdcard/Music directory. But, I sometimes download music to other locations, and perhaps you do too. Let's instead limit the remote to mp3 and ogg files:

    git-annex initremote android type=adb androiddirectory=/sdcard encryption=none exporttree=yes importtree=yes
    git annex wanted android 'include=*.mp3 or include=*.ogg'

The annex-tracking-branch can be the same as before, to limit the files that are synced to those in an android directory:

    git config remote.android.annex-tracking-branch master:android

And then do an initial sync:

    git annex sync --content android

Now, you can copy music and podcasts you want to listen to over to the Android device, by first copying them to the android directory of your git-annex repo:

    cp -a podcasts/LibreLounge/Episode_14__Secure_Scuttlebutt_with_Joey_Hess.ogg android/
    git annex add android
    git annex sync --content android

That will also import any new sound files from the Android device into your git-annex repo.

Once you're done with listening to something on the Android device, you can simply delete it from the device, and the next time git-annex syncs, it will get removed from the android directory. Or, you can delete it from the android directory and the next sync will delete it from the Android device.

comment 9

@g, when you have a problem like this, don't post a comment to a random place. File a bug report!

In this case, someone else has filed a bug: adb remote fails to import treeish

Comment by joey — Tue Jul 22 18:11:47 2025
adb sync problem (after some upgrades)

Hello, some time ago (it was Jan 2022) I set up an adb remote like explained here and I was able to sync multiple times with my phone.

After some time with no syncs, today I'm trying to sync again but I get this error:

    [...]
    list android_samsung_S4_01 find: bad arg '-printf'
    stat: Needs 1 argument (see "stat --help")

    git-annex: Unable to list contents of android_samsung_S4_01: adb find failed
    failed

    Merge made by the 'ort' strategy.
    (recording state in git...)
    sync: 1 failed

My doubt is that something changed when I upgraded git-annex, this is my currently installed version:

    git-annex version: 10.20220504
    build flags: Assistant Pairing Inotify TorrentParser MagicMime Feeds Testsuite S3 WebDAV
    dependency versions: aws-0.22 bloomfilter-2.0.1.0 cryptonite-0.29 DAV-1.3.4 feed-1.3.2.0 ghc-8.10.7 http-client-0.7.11 persistent-sqlite-2.13.0.3 torrent-10000.1.1 uuid-1.3.15
    key/value backends: SHA256E SHA256 SHA512E SHA512 SHA224E SHA224 SHA384E SHA384 SHA3_256E SHA3_256 SHA3_512E SHA3_512 SHA3_224E SHA3_224 SHA3_384E SHA3_384 SKEIN256E SKEIN256 SKEIN512E SKEIN512 BLAKE2B256E BLAKE2B256 BLAKE2B512E BLAKE2B512 BLAKE2B160E BLAKE2B160 BLAKE2B224E BLAKE2B224 BLAKE2B384E BLAKE2B384 BLAKE2BP512E BLAKE2BP512 BLAKE2S256E BLAKE2S256 BLAKE2S160E BLAKE2S160 BLAKE2S224E BLAKE2S224 BLAKE2SP256E BLAKE2SP256 BLAKE2SP224E BLAKE2SP224 SHA1E SHA1 MD5E MD5 WORM URL X*
    remote types: git gcrypt p2p S3 bup directory rsync web bittorrent webdav adb tahoe glacier ddar git-lfs httpalso borg hook external
    operating system: linux x86_64
    supported repository versions: 8 9 10
    upgrade supported from repository versions: 0 1 2 3 4 5 6 7 8 9 10
    local repository version: 8

Please do you have any hint on what I should do to make this running again?

Thanks! Giovanni

Comment by g — Tue Jul 22 18:11:47 2025
"mirror" workflow: usage of "wanted" expression is inconsistent with regular annex remote

Desired workflow with "adb":

- my repo of books is large :: dld/ new/ old/ read/ ... (so can't sync whole repo 1:1 to phone and must exclude old/ directory)
- usually I download books on phone, rarely on PC, and place them into dld/, then promising ones are moved into new/ and then read/ and each month lukewarm remainings in dld/ are moved into old/$date
- so files are never edited but moved extensively between folders -- and books added from both phone and PC are expected to have the same backend (currently PC uses SHA256 but why phone is unconditionally prefers SHA1 ???)
- this repo also contains textual journal with recorded progress (so I set largefiles to not mimefile=text/ and failed Mixed_content_repos_with_import_and_export)
- expecting to fully mirror hierarchy of files and its changes on both phone and PC (therefore I use "master" instead of "master:android" to prevent manually syncing duplicate hierarchy from subdir)
- expecting to NOT copy unwanted old/ files from PC to phone when exporting (works OK)
- expecting to auto-move all old/ files from phone to PC when importing (like usual "wanted" expression for "group=source" -- but they are ignored instead)
- expecting "delete" on phone will delete file only in "android/master" BUT not in "master" -- e.g. add symlinks back in merge commit when doing "git annex sync --content --not-only-annex" like described in "import after limited export"

Concerns:

- FAIL:(special-remote): using "git annex wanted android 'exclude=old/*'" ignores present files in old/ directory on phone and deletes all old/ symlinks on PC after sync.
- BUT:(git-annex): proper remote with same "wanted" expression behaves differently -- it tranfers old/ files content to other remotes and then drops them in current one (but keeps symlink untouched, not deleted like above)
- Why "import after limited export" is marked "done" if unwanted files are deleted? Or is it my misconception? https://git-annex.branchable.com/todo/export_preferred_content/

So, maybe expressions for "wanted content on remote" and "completely ignored in ALL remotes" must be two different options? However the current behavior of this "ignored" thing is too much counter-intuitive, making its existence questionable. Otherwise how can I accomplish abovementioned workflow i.e. make "adb special remote" to visibly behave during sync as regular annex repo w/o listed quirks?

Comment by amerlyq+annex — Tue Jul 22 18:11:47 2025
comment 6

This tip seems to work just fine when I try it. After `git annex sync --content android`, it had merged the remotes/android/master branch in. So I don't think there's a general problem with this tip that needs to be discussed here.

The remote tracking branch is created or updated every time git-annex imports from the android remote, eg every time that sync command is run.

If it's not creating it for you, you might have run into a bug, but if so you'll need to work out how others can reproduce it, and file a bug report.

Comment by joey — Tue Jul 22 18:11:47 2025
comment 5

How do I manually create this android_cable/master remote branch?

For the sake of testing, I created another remote on the same smartphone, and indeed, if "import" finishes in one try, there is a

       $ cat refs/remotes/android_test_cable/master
       8d7ada85897ef7df05bcdf1f2464ca678e03bd3b

How do I find an object that corresponds to the original `android_cable` special remote?

Comment by https://dead-doomer.livejournal.com/ — Tue Jul 22 18:11:47 2025
comment 4

Well, how would I know that that was significant? I resumed the import after interrupting it and let it run till it finished (several days actually).

In fact, I let "import" run several times till the end, the first time taking a lot of time, and the next ones taking ~10 minutes each.

Indeed, there seems to be no android_cable/master branch

    $ git --no-pager branch -ra
      git-annex
      synced/master

I tried to create the branch manually, but there seems to be no `git-annex branch`.

In addition, this post is easily googleable, whereas "forum" is not. I believe that documenting a typical-of-the-typical issue that a user may encounter at the place which if the point of the first consultation is fairly useful.

Comment by https://dead-doomer.livejournal.com/ — Tue Jul 22 18:11:47 2025
comment 2

At which point should I do ordinary git commits?

So far the repo has been at just the "Initial commit"

Comment by https://dead-doomer.livejournal.com/ — Tue Jul 22 18:11:47 2025
Importing specific directories from sdcard and internal storage
Do I understand correctly, that if I want to import/sync specific directories from both sdcard and internal storage, I effectively need to make two remotes (one for the sdcard, one for the internal storage)? I tried with `androiddirectory=/`, then `git annex wanted thephone 'include=/storage/self/primary/DCIM and include=/storage/33A8-601A/DCIM'` (I guess a trailing `/*` would have been necessary in addition), but that gives a gigantic amount of `find: [...] read/permission denied` errors upon import (I canceled it) and I guess as it traverses the entire file tree, it is very inefficient.
Comment by nobodyinperson — Tue Jul 22 18:11:47 2025
comment 1

`git-annex import master --from android` should have created an android/master branch with the imported files in it. There is apparently no such branch in your case.

But I see that you also posted strong> where you talk about having interrupted the import before it was complete. Since the branch only gets created when the import is allowed to finish, I think that explains your problem.

It's rarely a good idea to not fully explain what you actually did when reporting what seems like a problem. If I had not happened to read that forum post first, and noticed it was also from you, we would have embarked on a snipe hunt to find some bug that doesn't really exist.

Comment by joey — Tue Jul 22 18:11:47 2025
How do I specify a merge command if I don't need separate branches?

> > If you want its files to not be in a subdirectory, set it to "master" instead.

I did that. Than I issued:

    git annex import master --from android

which downloaded a lot of files

Then: `git annex merge android/master merge android/master merge: android/master - not something we can merge failed git-annex: merge: 1 failed`

I tried different combinations of android/master, ./master, master/. , but all seem to fail.

plain git status shows a lot of non-added files for an "initial commit", and no files added. git annex status lists a lot of

    ? sdcard/bla

files.

Any suggestions?

Comment by https://dead-doomer.livejournal.com/ — Tue Jul 22 18:11:47 2025
Comments on this page are closed.