# How to import multiple device classes to the Catalogue

Intended audience: administrators

The Device Classes Catalogue is available on the Tango Controls web page: http://www.tango-controls.org/.

To import multiple device classes to the catalogue from a repository please use an import script.

## Requirements

- Python2.7

- Subversion

- SVN python library: pip install svn

  > Warning

  > The library will not work if any of files in the SVN has no author defined which is the case for tango-ds repository on the Sourceforge. To avoid problems one can edit `svn/common.py` around line 358 to have something like the following:

  >     author = ''
  >     if commit_node.find('author') is not None:
  >         author = commit_node.find('author').text

- urllib2 library if it is not installed with Python itself: pip install urllib2

- Requests library version >= 2.12, you may need to run pip with –upgrade option: pip install --upgrade Requests

## How-to import multiple classes

1. Install packages listed in Requirements if not yet installed

2. Create a folder for a local copy of a repository:

     - example: mkdir ~/tmp/local-repo

3. Clone the import utility with git:

     - git clone https://github.com/piogor/dsc-import.git

4. Get into cloned sources:

     - cd dsc-import

5. Make your local branch to be sure your settings will not be overwritten by someone else.

     - git checkout -b my_local_branch

6. Update variables in a file `dsc_import_utility.py` to reflect your environment:

   >     FORCE_UPDATE = False  # when True no timestamps are checked and updates are performed
   >     TEST_SERVER_AUTH = False  # Set true if script is run against test server with additional authentication (webu test)
   >     VERIFY_CERT = False  # set this to false if running aginst test server without a valid certificate
   >     USE_DOC_FOR_NON_XMI = True # when True, parse documentation to get xmi conntent for device servers without XMI
   >     ADD_LINK_TO_DOCUMENTATION = True # when True it provides a link to documentation

   >     # set the following variables to point to the repositories
   >     LOCAL_REPO_PATH = '/home/piotr/tmp/tango-ds-repo/'  # local copy of the repository will be synced there
   >     LOG_PATH = '/home/piotr/tmp'  # where to log some information about import process, not used now.

   >     REMOTE_REPO_HOST = 'svn.code.sf.net'  # host of the SVN repository
   >     REMOTE_REPO_PATH = 'p/tango-ds/code'  # path within the server where the repository is located

   >     # if one would like to limit a search tree (useful for one device server update and/or tests)
   >     REPO_START_PATH = 'DeviceClasses'  # do not provide start nor end slashes

   >     # Tango Controls or test server address
   >     SERVER_BASE_URL = 'http://www.tango-controls.org/'

7. run with a command: python dsc_import_utility.py

   > Note

   > It will ask you for your credentials for tango-controls.org and import/update device classes using provided account.

## How the script works

It does import in the following way:

- It makes a local copy (in path defined by LOCAL_REPO_PATH) of a SVN repository to speed up search procedure.

- Then it searches the local copy for folders containing .XMI files. It takes into account the standard branches/tags/trunk structure. The folders where it findes .xmi files or a proper structure are listed as candidates to be device servers.

- Then, the list of candidates then is processed and compared (by repository URL) with content in the Device Classes Catalogue.

  - If there are changes or FORCE_UPDATE is True the catalogue is updated

    - For device server without .XMI file it looks for documentation server and tries to parse html documentation generated by Pogo.

  - If there ara no changes the device server is skipped

These information you will find in a `README.rst` ( https://github.com/piogor/dsc-import/blob/master/README.rst ).