Davix Documentation

Author
  Devresse Adrien ( adrie.nosp@m.n.de.nosp@m.vress.nosp@m.e@ce.nosp@m.rn.ch )

Developped at CERN (IT-SDC-ID)

Official WebSite: Here

User Documentation: Here

Mailing list : davix.nosp@m.-dev.nosp@m.el@ce.nosp@m.rn.c.nosp@m.h

## davix API :

file / object store API : Davix::DavFile
posix compatiblity layer : Davix::DavPosix
http request layer: Davix::HttpRequest
main header: davix.hpp

## What is davix ?

davix is a library and a set of tools for remote I/O on resources with HTTP based protocols. It aims to be a simple, performant and portable I/O layer for Cloud and Grid Storages services.

Davix supports:

- HTTP, WebDav, Amazon S3
- SSL/TLS
- X509 client auth with proxy credential support
- Vector operations (Partial reads, multi-range, single range)
- Metalinks
- Redirections caching
- Webdav parsing
- Data management operations ( mkdir, rm, stat )

The Davix targets to:

- Be simple to use for simple use cases
- Provide the needed features for High Performance I/O use cases
- Be a data management swiss knife for HTTP based data stores

## Examples

### Query basic file metadata

StatInfo infos;
DavFile file(context, "http://my.webdav.server.org/myfolder/myfile");
file.statInfo(NULL, infos);
std::cout << "my file is " << infos.size << " bytes large " << std::endl;

### Create a directory

DavFile file(context, "http://my.webdav.server.org/myfoldier/newfolder");
// creat directory
file.makeCollection(NULL);

### Get a full file content

DavFile f(context, "http://mysite.org/file");
int fd = open("/tmp/local_file", O_WRONLY | O_CREAT);
// get full file
file.getToFd(NULL,fd, NULL) < 0)

### Execute a partial GET

char buffer[255] = {0};
DavFile file(context, "http://mysite.org/file");
// get 100 bytes from http://mysite.org/file after an offset of 200 bytes
file.readPartial(NULL, buffer, 100, 200);

### Execute a Vector Operation

char buffer[255] = {0}
DavFile file(context, "http://mysite.org/file");
DavIOVecInput in[3];
DavIOVecOutput ou[3];
// setup vector operations parameters
// --------
// execute query
file.readPartialBufferVec(NULL, in, out , 3 , NULL);

### Random I/O in posix mode

Davix::DavPosix p;
// read ops
fd= p.open(NULL, "https://mywebdav-server.org/myfile.jpg", O_RDONLY, NULL);
p.read(fd, buffer, size, NULL);
p.pread(fd, buffer, size2, offset, NULL);
p.close(fd);
//
Davix::DavPosix
POSIX-like API of Davix.
Definition davposix.hpp:63
Davix::DavPosix::close
int close(DAVIX_FD *fd, DavixError **err)
close a existing file descriptor.
Davix::DavPosix::read
ssize_t read(DAVIX_FD *fd, void *buffer, size_t count, DavixError **err)
read a file in a POSIX-like approach with HTTP(S).
Davix::DavPosix::open
DAVIX_FD * open(const RequestParams *params, const std::string &url, int flags, DavixError **err)
open a file for read/write operation in a POSIX-like approach.
Davix::DavPosix::pread
ssize_t pread(DAVIX_FD *fd, void *buffer, size_t count, off_t offset, DavixError **err)
do a partial read of a file in a POSIX-like approach with HTTP(S).

### Manual HTTP query:

Davix::HttpRequest req("https://restapi-server.org/rest")
req.addHeaderField(...)
req.setRequestMethod("PUT")
// .. configure ....
//
//
// execute your request
req.executeRequest(...);
Davix::HttpRequest
Http low level request interface.
Definition httprequest.hpp:87

## How to compile

- Davix Dependencies :
  - openssl
  - libxml-2.0
  - Doxygen ( optional, for documentation generation )
- Davix Portability :
  - Target any POSIX OS
  - Ported on Linux > 2.6 , Windows with Cygwin, OSX > 10.2, AIX.
  - Packaged on Debian > 6, Ubuntu > 13.04, Fedora > 18, SL > 5,
- Compile :
  - " 1. git clone http://git.cern.ch/pub/davix "
  - " 2. cd davix "
  - " 3. mkdir build; cd build"
  - " 4. cmake ../"
  - " 5. make "
- Generate doc :
  - make doc
- Compile and run unit tests :
  - cmake ../
  - make
  - make test
- Compile & execute func tests : warning : functionals test needs davserver and a valid credential
  - " 4. cmake -DFUNCTIONAL_TESTS=TRUE ../ "
  - " 5. . ../test/setup_test_env.sh - " 5. make; make test"
- make RPMS :
  - ./packaging/make-srpm.sh

## Play with davix command line tool :

davix has a set of command line tools for testing purpose and demonstration

    -> davix-ls: file listing
    -> davix-get: download operations
    -> davix-put: upload operations
    -> davix-http: low level query composition

## TODO in Davix

    - WebHDFS support
    - S3 ACL support
    - CDMI support
    - X-stream mode support ( metalink multi source download )

    For any contribution please contact us on davix-devel@cern.ch