| Network Configuration |

PubMed | Entrez | BLAST | OMIM | Taxonomy | Structure

The Switch

Old Dispatcher

New Dispatcher

Firewalling

Work Arounds

Additional Info

# NCBI Dispatcher Parameters

There are two pieces of code in C toolkit that allow an application to contact NCBI dispatchers, the new and the old one. The old dispatcher is being phased out and is no longer supported at NCBI. All new services that are added get configured with the new dispatcher. The code to access the new dispatcher had been placed in the toolkit a while ago, and should be ready for use in all applications compiled and linked against the toolkit released since beginning of 2002 [recent repository check showed that actually the cut-off date can perhaps be half a year earlier, in early summer 2001]. Older applications have to be upgraded by re-compiling/re-linking in order to take advantage of the new code.

In the beginning of August 2002, the switch was made in the toolkit's code that now directs all network requests to go through the new dispatcher. So all applications linked against the toolkit at a later date should be using the new dispatcher automatically.

Below is the description of parameters and differences between the two dispatchers, and instructions on how to set an existing application to use the new dispatcher's code without rebuilding.

Although the dispatcher should work behind the scenes absolutely transparently for the user, some applications may fail if for example there are on-site configurational discrepancies between the two dispatchers. In fact, every attempt was made to keep the configurations consistent at the time of the switch, but still there is a little chance that something could have been missed.

## The switch

All `SRV_...` parameters, outlined below, can be set in either program environment prior to the application run (using "`setenv`" command in `[t]csh`, or "`export`" command in [ba]sh), or in the section `[NET_SERV]` of the `.rc` (or `.ini`) file, also known as a configuration (or registry) file of the application.
Note: Settings in the environment always take precedence and override those read from the registry file.
Note: On MS-Windows, the registry file usually resides in the master Windows directory, which is either `WINDOWS` or `WINNT`.

A dispatcher to use can be controlled by the environment variable `SRV_CONN_MODE` as follows:

`SRV_CONN_MODE=SERVICE` | chooses the new dispatcher [default for all programs built since mid August, 2002]
`SRV_CONN_MODE=WWW` | chooses the old dispatcher, without firewalling [this was the default prior to the switchover in mid August, 2002]
`SRV_CONN_MODE=FIREWALL` | chooses the old dispatcher, with firewalling

## Parameters of the OLD dispatcher

The parameters, which control the OLD dispatcher (both with and without firewall), are in the table below. Please note that they can appear both in the environment or in the registry file (section `[NET_SERV]`), with those found in the environment taking the precedence.

`SRV_ENGINE_HOST`
`SRV_ENGINE_PORT`

host and port to locate dispatcher. Default = `www.ncbi.nlm.nih.gov:80` (should normally not be changed)

`SRV_ENGINE_URL` | path part of URI to locate dispatcher. Default = `/Service/nph-dispd.cgi` (should normally not be changed, either)
`SRV_CONN_TIMEOUT` | time (as a floating point value, seconds) to wait for network activity, at most. Default = 30. Special value `INFINITE` is also recognized to set the timeout value to "wait forever". Note: If you are in the United States, the default timeout of 30 seconds should suffice. From foreign countries with poor Internet connections to the U.S., you may want to increase this value.
`SRV_CONN_TRY` | how many connection attempts to perform if unsuccessful, default = 3

`SRV_HTTP_PROXY_HOST`
`SRV_HTTP_PROXY_PORT`

host and port of transparent HTTP proxy (if in use at the client's end), default = none

`SRV_PROXY_HOST` | CERN-like (non-transparent) proxy host (if in use at the client's end), default = none
`SRV_DEBUG_PRINTOUT` | switch to print debugging/trace output, default = NO Activate printing with YES, 1, or TRUE.
`NI_SERVICE_NAME_<name>` | reassignment of the service name, e.g.: `NI_SERVICE_NAME_ID1 ID1Prod` used for temporary name override, if a developer instructed you to do so (usually, neither required nor advised). Note: Service `name` must be all capitalized for this setting to work, e.g. "Entrez" becomes `ENTREZ`.

Atlernate service name can also be specified via the registry file by defining they key `SERVICE_NAME` with desired value, which should be located in the section having the service name in all capitals. The following registry fragment changes the name of the `Entrez` service:

    [ENTREZ]
    SERVICE_NAME=EntrezTest

## Parameters of the NEW dispatcher

For the sake of backward compatibility, all the above (`SRV_...`) parameters are accepted. They could be either in the environment or in the registry file, section `[NET_SERV]`.

The following additional parameters are also recognized (and are the preferred way to tune up the new dispatcher). These parameters can be specified in the environment as printed in the table, or they can be read from the registry file's section `[CONN]`, but then without the prefix `CONN`. For example, the environment setting `CONN_FIREWALL=TRUE` is equivalent to the following registry file fragment:

    [CONN]
    FIREWALL=TRUE

There are also per-service settings, taking the form `<NAME>_CONN_<PARAMETER>` in the environment, or just like printed below if found in the section `[<NAME>]` in the registry file. For example, the following settings of `FIREWALL` in the environment and the registry file are equivalent, and affect `Bounce` service only (leaving all other services with default or their individual settings, if any):

environment:

    BOUNCE_CONN_FIREWALL=TRUE

registry file:

    [BOUNCE]
    CONN_FIREWALL=TRUE

The order of significance (from the most to the least) of specifications is as follows:

1. Per-service environment variable, e.g. `BOUNCE_CONN_FILREWALL=TRUE`;
2. Per-service registry key, e.g. `CONN_FIREWALL=TRUE` in the section `[BOUNCE]`;
3. Common environment setting, e.g. `CONN_FIREWALL=TRUE`;
4. Common registry key from section `[CONN]`, e.g. `FIREWALL=TRUE`;
5. Default setting.

`CONN_HOST`
`CONN_PORT`

see `SRV_ENGINE_HOST` and `SRV_ENGINE_PORT` above.

`CONN_PATH` | see `SRV_ENGINE_URL`, default = `/Service/dispd.cgi`
`CONN_ARGS` | arguments for service/dispatcher, going after the '?' in CGI's parameters, default = none
`CONN_REQ_METHOD` | HTTP request method to use (default = `POST`, may also be one of `ANY` and `GET`).
`CONN_TIMEOUT` | see `SRV_CONN_TIMEOUT`.
`CONN_MAX_TRY` | see `SRV_CONN_TRY`.

`CONN_HTTP_PROXY_HOST`
`CONN_HTTP_PROXY_PORT`

see `SRV_HTTP_PROXY_HOST` and `SRV_HTTP_PROXY_PORT`.

`CONN_PROXY_HOST` | see `SRV_PROXY_HOST`.
`CONN_DEBUG_PRINTOUT` | see `SRV_DEBUG_PRINTOUT`. Additional value `DATA` is provided to print the network data as being transferred at the transport [TCP] layer.
`CONN_STATELESS` | `TRUE` or `FALSE`, default = `FALSE`. A proper value has to be set depending on the nature of the client (Web-browsers are `CONN_STATELESS=TRUE`). For dispatching by default, this variable is not to be touched!
`CONN_FIREWALL` | `TRUE` or `FALSE`, default = `FALSE`. A proper value has to be chosen depending on whether the client is behind a firewall or not. Use `TRUE` if so! This parameter of the new dispatcher is analogous to switching the old dispatcher into firewall mode by specifying `FIREWALL` in `SRV_CONN_MODE`.
`<name>_CONN_SERVICE_NAME` | see description for `NI_SERVICE_NAME_...` above.

When intermixed with older (`SRV_...`) parameters, the order of consideration is the following: the new parameters (`CONN_...`) first as described in the table above, then `SRV_...` parameters, thus all `SRV_...` parameters taking precedence over all `CONN_...` parameters. The only exception is the service name override (`{..._CONN|NI}_SERVICE_NAME_...`), which works the different way:

- All other parameters loaded, first `CONN_...` using the original service name as specified in the application (e.g. `Entrez`) from the registry/environment, followed by `SRV_...` found in the registry/environment.
- `SERVICE_NAME` registry key in the service's section followed by `NI_SERVICE_NAME_...` environment variable (if exist) are then consulted for an alternate name.
- `..._CONN_SERVICE_NAME` environment variable (or service's registry section with `SERVICE_NAME` key) in then consulted, using a new name from the substitution above (if occurred). This process is recursive but with no loop detection, i.e. A_CONN_SERVICE_NAME=B and B_CONN_SERVICE_NAME=A would create an infinite loop, which breaks the application. The following registry/environment combination changes the name of the `Entrez` service into `Entrez2`:

  environment:

      NI_SERVICE_NAME_ENTREZ=EntrezTest

  registry:

      [ENTREZ]
      SERVICE_NAME=Entrez1 ; Note that this key is overriden in the env

      [ENTREZ_TEST]
      SERVICE_NAME=Entrez2

Note that if a client connects to new NCBI dispatcher, then the dispatcher might be able to switch the client into pseudo-firewall mode, should the dispatcher notice that this might be needed for the client to further succeed with the service requested. This however is not a substitute for `CONN_FIREWALL` parameter, and an application, which connects from behind a firewall, must still use `CONN_FIREWALL=TRUE` in order to function flawlessly.

Also note that C toolkit can have debugging printouts disabled in the core, thus `{SRV|CONN}_DEBUG_PRINTOUT` will have no or very little effect. In order to override the core settings, environment setting `DEBUG_PRINTOUT=TRUE` could be used. Unlike `{SRV|CONN}_DEBUG_PRINTOUT` the last key has no effect outside the environment (and thus specifying `DEBUG_PRINTOUT` in the registry file will not affect in-core logging settings). This override option was recently added to the toolkit, and applications built before mid August 2002 lack this feature.

For applications linked against C++ toolkit, environment settings `DIAG_POST_LEVEL=0` and `DIAG_TRACE=1` would enable the debugging printouts regardless of the in-core settings. You may also use these keys in the `[DEBUG]` section of the registry file (but not in the `[NET_SERV]` or `[CONN]` sections).

# Taking care of firewall connections

If you are behind a firewall, it must be configured correctly to access NCBI services. Your network administrators may have done this already. If not, please have them read the section below.

The following section is intended for network administrators:

Using NCBI services from behind a security firewall requires opening ports in your firewall. The ports to open are:

    Firewall Port        IP Address
    --------------------------------
     5860..5870         130.14.29.112
     5860..5870         165.112.7.12

If your firewall is not transparent, the firewall port number should be mapped to the same port number on the external host.

While not every port from the table above might be currently in use, we stongly recommend you to have them all configured on your firewall.

TROUBLESHOOTING: You can test whether these ports are accessible from your host by just running, for example (see the "Ports to open" list above):

      telnet 130.14.29.112 5861

and entering a line of arbitrary text in the telnet session. If everything is fine, your TELNET session would look as follows (the line "test" is your input there):

    | > telnet 130.14.29.112 5861
    | Trying 130.14.29.112...
    | Connected to 130.14.29.112.
    | Escape character is '^]'.
    | test
    | NCBI Firewall Daemon:  Invalid ticket. Connection closed.
    | Connection closed by foreign host.

There is also an auxiliary UNIX shell script fwd_check.sh to check all of the above addresses.

To check the ports as they are seen from within NCBI you can use this test page, which shows you whether the ports are open and the firewall daemon is listening on them. The page does not however provide you with the information whether these ports are accessible from your site. You still have to use fwd_check.sh to check that.

# Working around connectivity problems

Executable `Psrventr.REAL`, which implements old `Entrez` service on Solaris sparc, was confirmed flaky, and had to be removed from configurations of both new and old dispatchers on the sparcs.

Although this should not affect the users of the new dispatcher (as the service had been moved to and is currently provided from a number of Intel machines), the users of the old dispatcher (which was not ported to `Linux/Intel` platform, and overall is considered for phasing out as inflexible and outdated) may be affected.

The workaround is to use the code in the toolkit that has been around long ago but was not made active until mid August 2002 - we then switched the toolkit to use the new dispatcher code by default.

For that, the user has to define `SRV_CONN_MODE=SERVICE` either in the environment, or in the configuration (`.ini`) file, section `[NET_SERV]`, and can do so without an absolute necessity of downloading the new toolkit code from NCBI and rebuilding. Only very-very old applications will not be able to use the switch, and have to be upgraded by linking against freshened libraries.

If an application runs from behind a firewall, then `CONN_FIREWALL=TRUE` should also be set in the environment (in addition to `SRV_CONN_MODE=SERVICE`). Please note that `CONN_FIREWALL` must remain set even if you switch into the newer toolkit release, which does not require setting `SRV_CONN_MODE` for the use of the new dispatcher.

The overall rule in the case when your application fails to connect to an NCBI service is to undertake the following sequence of steps:

1. Try to force your application to use the new dispatcher code by setting `SRV_CONN_MODE` to `SERVICE` in the environment/registry;
2. If this does not help, or if you are connecting from behind a firewall, try to set `CONN_FIREWALL` to `TRUE` in addition to the above setting, and run your application again;
3. If you are still unable to connect, set `SRV_DEBUG_PRINTOUT` to `TRUE`, and collect the output. (You may also need to set environment variable `DEBUG_PRINTOUT` to `TRUE` in addition, if the in-core settings prevent debugging output from being released onto the terminal. Please note that unlike other `{CONN|SRV}...` keys described in this document, `DEBUG_PRINTOUT` is honored only if set in the environment and has no effect if put in the registry file.) Beware that debugging output may be copious.
   - If with all the above settings you can find references to either `Service/nph-dispd.cgi` or `Service/nph-ncbid.cgi` in the collected output, it means that your application was linked against very old version of the toolkit and has to be rebuilt;
   - If you cannot find references to either of these CGIs, then you can consider contacting NCBI Help Desk to resolve your problem.

Small reference on how to set the environment in various shells (to know the shell in UNIX/Linux, execute "`echo $SHELL`"):

- C-like shells: `csh` or `tcsh`

          setenv SRV_DEBUG_PRINTOUT TRUE

- Born-like shells: `sh`, `ash`, `ksh`, `zsh`, or `bash`

          SRV_DEBUG_PRINTOUT=TRUE; export SRV_DEBUG_PRINOUT

  In `ksh`, `zsh`, and `bash` the two commands above can be combined:

          export SRV_DEBUG_PRINTOUT=TRUE

- DOS shell (MS-Windows/DOS): `command.com` or `cmd.exe`

          set SRV_DEBUG_PRINTOUT=TRUE

# Additional Information

Note: Very old NCBI clients used different application configuration settings and ports than listed above. These clients are no longer supported by NCBI.

More technical and brief note on dispatcher selection and parameters is in `network/nsclilib/readme` located in the C toolkit source tree.

Description of the new NCBI dispatcher can be found here.

Discussion of firewall settings can be found here and also provided in directory `doc` of the C toolkit.

If you have a problem, which you are unable to resolve on your own, or you have a question, please contact `info@ncbi.nlm.nih.gov` for further information.

Source: distrib/doc/dispatcher.html | $Date: 2011/09/06 22:30:30 $ | $Revision: 6.29 $