TEST_UTIL(3)

NAME
nfstest.test_util - Test utilities module

DESCRIPTION

Provides a set of tools for testing either the NFS client or the NFS server, most of the functionality is focused mainly on testing the client. These tools include the following:

    - Process command line arguments
    - Provide functionality for PASS/FAIL
    - Provide test grouping functionality
    - Provide multiple client support
    - Logging mechanism
    - Debug info control
    - Mount/Unmount control
    - Create files/directories
    - Provide mechanism to start a packet trace
    - Provide mechanism to simulate a network partition
    - Support for pNFS testing

In order to use some of the functionality available, the user id in all the client hosts must have access to run commands as root using the 'sudo' command without the need for a password, this includes the host where the test is being executed. This is used to run commands like 'mount' and 'umount'. Furthermore, the user id must be able to ssh to remote hosts without the need for a password if test requires the use of multiple clients.

Network partition is simulated by the use of 'iptables', please be advised that after every test run the iptables is flushed and reset so any rules previously setup will be lost. Currently, there is no mechanism to restore the iptables rules to their original state.

CLASSES

  class TestUtil(nfstest.nfs_util.NFSUtil)
TestUtil(**kwargs)

TestUtil object

TestUtil() -> New server object

Usage:
    x = TestUtil()

    # Process command line options
    x.scan_options()

    # Start packet trace using tcpdump
    x.trace_start()

    # Mount volume
    x.mount()

    # Create file
    x.create_file()

    # Unmount volume
    x.umount()

    # Stop packet trace
    x.trace_stop()

    # Exit script
    x.exit()

Methods defined here:
---------------------

__del__(self)
Destructor

Gracefully stop the packet trace, cleanup files, unmount volume,
and reset network.

__init__(self, **kwargs)
Constructor

Initialize object's private data.

    sid:
        Test script ID [default: '']
        This is used to have options targeted for a given ID without
        including these options in any other test script.

    usage:
        Usage string [default: '']

    testnames:
        List of test names [default: []]
        When this list is not empty, the --runtest option is enabled and
        test scripts should use the run_tests() method to run all the
        tests. Test script should have methods named as <testname>_test.

    testgroups:
        Dictionary of test groups where the key is the name of the test
        group and its value is a dictionary having the following keys:
        tests:
        A list of tests belonging to this test group
        desc:
        Description of the test group, this is displayed
        in the help if the name of the test group is also
        included in testnames
        tincl:
        Include a comma separated list of tests belonging to
        this test group to the description [default: False]
        wrap:
        Reformat the description so it fits in lines no
        more than the width given. The description is not
        formatted for a value of zero [default: 72]

    Example:
        x = TestUtil(testnames=['basic', 'lock'])

        # The following methods should exist:
        x.basic_test()
        x.lock_test()

cleanup(self)
Clean up test environment.

Remove any files created: test files, trace files.

close_files(self, *fdlist)
Close all files opened by open_files() and all file descriptors
given as arguments.

compare_data(self, data, offset=0, pattern=None, nlen=32, fd=None, msg='')
Compare data to the given pattern and return a three item tuple:
absolute offset where data differs from pattern, sample data at
diff offset, and the expected data at diff offset according to
pattern. If data matches exactly it returns (None, None, None).

    data:
        Data to compare against the pattern

    offset:
        Absolute offset to get the expected data from pattern
        [default: 0]

    pattern:
        Data pattern function or string. If this is a function,
        it must take offset and size as positional arguments.
        If given as a string, the pattern repeats over and over
        starting at offset = 0 [default: self.data_pattern]

    nlen:
        Size of sample data to return if a difference is found
        [default: 32]

    fd: Opened file descriptor for the data, this is used where
        the data comes from a file and a difference is found right
        at the end of the given data. In this case, the data is
        read from the file to return the sample diff of size given
        by nlen [default: None]

    msg:
        Message to append to debug message if a difference is
        found. If set to None, debug messages are not displayed
        [default: '']

compare_mount_args(self, mtopts1, mtopts2)
Compare mount arguments

config(self, msg)
Display config message and terminate test with an exit value of 2.

create_dir(self, dir=None, mode=493)
Create a directory under the given directory with the given mode.

create_file(self, offset=0, size=None, dir=None, mode=None, **kwds)
Create a file starting to write at given offset with total size
of written data given by the size option.

    offset:
        File offset where data will be written to [default: 0]

    size:
        Total number of bytes to write [default: --filesize option]

    dir:
        Create file under this directory

    mode:
        File permissions [default: use default OS permissions]

    pattern:
        Data pattern to write to the file [default: data_pattern default]

    ftype:
        File type to create [default: FTYPE_FILE]

    hole_list:
        List of offsets where each hole is located [default: None]

    hole_size:
        Size of each hole [default: --wsize option]

    verbose:
        Verbosity level [default: 0]

    dlevels:
        Debug level list to use [default: ["DBG2", "DBG3", "DBG4"]]

    Returns the file name created, the file name is also stored
    in the object attribute filename -- attribute absfile is also
    available as the absolute path of the file just created.

    File created is removed at cleanup.

create_rexec(self, servername=None, **kwds)
Create remote server object.

data_pattern(self, offset, size, pattern=None)
Return data pattern.

    offset:
        Starting offset of pattern

    size:
        Size of data to return

    pattern:
        Data pattern to return, default is of the form:
        hex_offset(0x%08X) abcdefghijklmnopqrst

delay_io(self, delay=None)
Delay I/O by value given or the value given in --iodelay option.

exit(self)
Terminate script with an exit value of 0 when all tests passed
and a value of 1 when there is at least one test failure.

get_dirname(self, dir=None)
Return a unique directory name under the given directory.

get_filename(self, dir=None)
Return a unique file name under the given directory.

get_logname(self, remote=False)
Get next log file name.

get_marker_index(self, marker_id=None)
Find packet index of the trace marker given by the marker id

    marker_id:
        ID of trace marker to find in the packet trace, if this is
        not given the current marker id is used [default: None]

get_name(self)
Get unique name for this instance.

insert_trace_marker(self, name=None)
Send a LOOKUP for an unknown file to have a marker in
the packet trace and return the trace marker id

    name:
        Use this name as the trace marker but the caller must make
        sure this is a unique name in order to find the correct
        index for this marker. This could also be used to add any
        arbitrary information to the packet trace [default: None]

lock_files(self, lock_type=None, offset=0, length=0)
Lock all files opened by open_files().

need_run_test(self, testname)
Return True only if user explicitly requested to run this test

open_files(self, mode, create=True)
Open files according to given mode, the file descriptors are saved
internally to be used with write_files(), read_files() and
close_files(). The number of files to open is controlled by
the command line option '--nfiles'.

The mode could be either 'r' or 'w' for opening files for reading
or writing respectively. The open flags for mode 'r' is O_RDONLY
while for mode 'w' is O_WRONLY|O_CREAT|O_SYNC. The O_SYNC is used
to avoid the client buffering the written data.

process_client_option(self, option='client', remote=True, count=1)
Process the client option

Clients are separated by a "," and each client definition can have
the following options separated by ":":
    client:server:export:nfsversion:port:proto:sec:mtpoint

    option:
        Option name [default: "client"]

    remote:
        Expect a client hostname or IP address in the definition.
        If this is set to None do not verify client name or IP.
        [default: True]

    count:
        Number of client definitions to expect. If remote is True,
        return the number of definitions listed in the given option
        up to this number. If remote is False, return exactly this
        number of definitions [default: 1]

    Examples:
        # Using positional arguments with nfsversion=4.1 for client1
        client=client1:::4.1,client2

        # Using named arguments instead
        client=client1:nfsversion=4.1,client2

process_option(self, value, arglist=[], typemap={})
Process option with a list of items separated by "," and each
item in the list could have different arguments separated by ":".

    value:
        String of comma separated elements

    arglist:
        Positional order of arguments, if this list is empty,
        then use named arguments only [default: []]

    typemap:
        Dictionary to convert arguments to their given types,
        where the key is the argument name and its value is the
        type function to use to convert the argument [default: {}]

read_files(self)
Read a block of data (size given by --rsize) from all files opened
by open_files() for reading.

remove_test(self, testname)
Remove all instances of test from the list of tests to run

run_func(self, func, *args, **kwargs)
Run function with the given arguments and return the results.
All positional arguments are passed to the function while the
named arguments change the behavior of the method.
Object attribute "oserror" is set to the OSError object if the
function fails.

    msg:
        Test assertion message [default: None]

    err:
        Expected error number [default: 0]

run_tests(self, **kwargs)
Run all test specified by the --runtest option.

    testnames:
        List of testnames to run [default: all tests given by --testnames]

    All other arguments given are passed to the test methods.

scan_options(self)
Process command line options.

Process all the options in the file given by '--file', then the
ones in the command line. This allows for command line options
to over write options given in the file.

Format of options file:
    # For options expecting a value
    <option_name> = <value>

    # For boolean (flag) options
    <option_name>

Process options files and make sure not to process the same file
twice, this is used for the case where HOMECFG and CWDCFG are the
same, more specifically when environment variable HOME is not
defined. Also, the precedence order is defined as follows:
  1. options given in command line
  2. options given in file specified by the -f|--file option
  3. options given in file specified by ./.nfstest
  4. options given in file specified by $HOME/.nfstest
  5. options given in file specified by /etc/nfstest

NOTE:
  Must use the long name of the option (--<option_name>) in the file.

set_nfserr_list(self, nfs3list=[], nfs4list=[], nlm4list=[], mnt3list=[])
Temporaly set the NFS list of expected NFS errors in the next call
to trace_open

setup(self, nfiles=None)
Set up test environment.

Create nfiles number of files [default: --nfiles option]

str_args(self, args)
Return the formal string representation of the given list
where string objects are truncated.

test(self, expr, msg, subtest=None, failmsg=None, terminate=False)
Test expr and display message as PASS/FAIL, terminate execution
if terminate option is True.

    expr:
        If expr is true, display as a PASS message,
        otherwise as a FAIL message

    msg:
        Message to display

    subtest:
        If given, append this string to the displayed message and
        mark this test as a member of the sub-group given by msg

    failmsg:
        If given, append this string to the displayed message when
        expr is false [default: None]

    terminate:
        Terminate execution if true and expr is false [default: False]

    If tverbose=normal or level 1:
        Sub-group message is displayed as a PASS/FAIL message including
        the number of tests that passed and failed within the sub-group
    If tverbose=verbose or level 2:
        All tests messages are displayed

test_description(self, tname=None)
Return the test description for the current test

test_group(self, msg)
Display heading message and start a test group.

If tverbose=group or level 0:
    Group message is displayed as a PASS/FAIL message including the
    number of tests that passed and failed within this test group.
If tverbose=normal|verbose or level 1|2:
    Group message is displayed as a heading messages for the tests
    belonging to this test group.

test_info(self, msg)
Display info message.

test_options(self, name=None)
Get options for the given test name. If the test name is not given
it is determined by inspecting the stack to find which method is
calling this method.

testid_count(self, tid)
Return the number of instances the testid has occurred.

trace_open(self, *kwts, **kwds)
This is a wrapper to the original trace_open method where the
packet trace is scanned for NFS errors and a failure is logged
for each error found not given on the list of expected errors
set with method set_nfserr_list. Scanning for NFS error is done
only if --nfserrors option has been specified.

trace_start(self, *kwts, **kwds)
This is a wrapper to the original trace_start method to reset
the trace marker state

verify_client_option(self, tclient_dict, option='client')
Verify the client option is required from the list of tests to run.
Also, check if enough clients were specified to run the tests.

    tclient_dict:
        Dictionary having the number of clients required by each test

    option:
        Option name [default: "client"]

verify_file_data(
    self,
    msg=None,
    pattern=None,
    path=None,
    filesize=None,
    nlen=None,
    cmsg=''
)
    Verify file by comparing the data to the given pattern.
    It returns the results from the compare_data method.

    msg:
        Test assertion message. If set to None, no assertion is
        done it just returns the results [default: None]
    pattern:
        Data pattern function or string. If this is a function,
        it must take offset and size as positional arguments.
        If given as a string, the pattern repeats over and over
        starting at offset = 0 [default: self.data_pattern]
    path:
        Absolute path of file to verify [default: self.absfile]
    filesize:
        Expected size of file to be verified [default: self.filesize]
    nlen:
        Size of sample data to return if a difference is found
        [default: compare_data default]
    cmsg:
        Message to append to debug message if a difference is
        found. If set to None, debug messages are not displayed
        [default: '']

warning(self, msg)
Display warning message.

write_data(
    self,
    fd,
    offset=0,
    size=None,
    pattern=None,
    verbose=0,
    dlevel='DBG5'
)
    Write data to the file given by the file descriptor

    fd:
        File descriptor
    offset:
        File offset where data will be written to [default: 0]
    size:
        Total number of bytes to write [default: --filesize option]
    pattern:
        Data pattern to write to the file [default: data_pattern default]
    verbose:
        Verbosity level [default: 0]

write_files(self)
Write a block of data (size given by --wsize) to all files opened
by open_files() for writing.

Static methods defined here:
----------------------------

get_list(value, nmap, sep=',')
Given the value as a string of 'comma' separated elements, return
a list where each element is mapped using the dictionary 'nmap'.
    nmap = {"one":1, "two":2}
    out = x.get_list("one", nmap)        # out = [1]
    out = x.get_list("one,two", nmap)    # out = [1,2]
    out = x.get_list("two,one", nmap)    # out = [2,1]
    out = x.get_list("one,three", nmap)  # out = None

str_list(value, vtype=<class 'str'>, sep=',')
Return a list of <vtype> elements from the comma separated string.

SEE ALSO
baseobj(3), formatstr(3), nfstest.host(3), nfstest.nfs_util(3), nfstest.rexec(3), nfstest.utils(3), packet.nfs.nfs3_const(3), packet.nfs.nfs4_const(3)

BUGS
No known bugs.

AUTHOR
Jorge Mora (mora@netapp.com)