SFSCHUNKSERVER.CFG(5)

NAME
sfschunkserver.cfg - main configuration file for *sfschunkserver*

DESCRIPTION

The file sfschunkserver.cfg contains configuration of SaunaFS chunkserver process.

SYNTAX

Syntax is:

OPTION = VALUE
    Lines starting with # character are ignored.

OPTIONS

Configuration options:

DATA_PATH
    where to store files with usage statistics and daemon lock file

LABEL
    the label of this chunkserver (for tiering)

WORKING_USER
    user to run daemon as

WORKING_GROUP
    group to run daemon as (optional - if empty then default user group will be used)

SYSLOG_IDENT
    name of process to place in syslog messages (default is sfschunkserver)

LOCK_MEMORY
    whether to perform mlockall() to avoid swapping out sfschunkserver process (default is 0, i.e. no)

LIMIT_GLIBC_MALLOC_ARENAS
    Linux only: limit glibc malloc arenas to given value - prevents from using huge amount of virtual memory. This can influence performance by reducing memory fragmentation and improving cache locality, but it may also lead to contention and reduced parallelism in multi-threaded applications. Use it in constrained memory environments, recommended values are 4 or 8. (default is 0: disabled or let glibc decide)

MALLOC_TRIM_INTERVAL
    interval in seconds at which the chunkserver will attempt to release unused memory back to the OS using malloc_trim(). This can help reduce the process’s memory footprint by returning free memory to the system more frequently. Set to 0 to disable memory trimming entirely. (default is 0, 0 = disabled)

NICE_LEVEL
    nice level to run daemon with (default is -19 if possible; note: process must be started as root to increase priority)

MASTER_HOST
    address of SaunaFS master host to connect with (default is sfsmaster)

MASTER_PORT
    number of SaunaFS master port to connect with (default is 9420)

CLUSTER_ID
    Only connect to metadata servers with this CLUSTER_ID. Prevents accidental connections to wrong metadata servers. (default is default)

MASTER_RECONNECTION_DELAY
    delay in seconds before trying to reconnect to the master server after disconnection (default is 5)

MASTER_TIMEOUT
    timeout (in seconds) for the master server connection (default is 60, minimum is 0.01)

MASTER_NR_OF_WORKERS
    number of threads that the connection to master may use to process operations on chunks, like create, duplicate, replicate and truncate (default is 10, minimum is 2)

MASTER_REPLICATION_NR_OF_WORKERS
    number of threads that the connection to master may use to process replications operations on chunks. (default is 5, minimum in 1)

BIND_HOST
    local address to use for connecting with the master server (default is *, i.e. default local address)

CSSERV_LISTEN_HOST
    IP address to listen on for client (mount) connections (* means any)

CSSERV_LISTEN_PORT
    port to listen on for client (mount) connections (default is 9422)

HDD_CONF_FILENAME
    alternative name of sfshdd.cfg file

HDD_LEAVE_SPACE_DEFAULT
    free space threshold to set volume as 100% utisaued when there is less than given amount of free space left (default is "4GiB"). This number is always added to the used disk space reported by chunkserver.

HDD_TEST_FREQ
    chunk test period in seconds (default is 10)

HDD_CHECK_CRC_WHEN_READING
    whether to check the CRC on every read operation (default is 1)

HDD_CHECK_CRC_WHEN_WRITING
    whether to check the CRC on every write operation. The safest and recommended value is 1, disable it only to test maximum throughput. (default is 1)

HDD_ADVISE_NO_CACHE
    whether to remove each chunk from page when closing it to reduce cache pressure generated by chunkserver (default is 0, i.e. no)

HDD_PUNCH_HOLES
    if enabled then chunkserver detects zero values in chunk data and frees corresponding file blocks (decreasing file system usage). This option works only on Linux with file systems supporting punching holes (XFS, ext4, Btrfs, tmpfs)

ENABLE_LOAD_FACTOR
    if enabled, chunkserver will send periodical reports of its I/O load to master, which will be taken into consideration when picking chunkservers for I/O operations.

REPLICATION_BANDWIDTH_LIMIT_KBPS
    limit how many kilobytes can be replicated from other chunkservers to this chunkserver in every second (by default undefined, i.e. no limits)

NR_OF_NETWORK_WORKERS
    number of threads which handle (in a round-robin manner) connections with clients (default is 4); these threads are responsible for reading from sockets and coping data from internal buffers to sockets

NR_OF_HDD_WORKERS_PER_NETWORK_WORKER
    number of threads that each network worker may use to do disk operations like opening chunks, reading or writing them (default is 16)

BGJOBSCNT_PER_NETWORK_WORKER
    maximum number of jobs that each network worker may use for disk operations (default is 4000)

POLL_TIMEOUT_MS
    Maximum amount of time in milliseconds that the polling operation will wait for events. In the chunkservers, the same value is applied for the events loop and for the network worker threads. Smaller values could reduce latency at the cost of CPU usage (default: 50)

MAX_BLOCKS_PER_HDD_WRITE_JOB
    maximum number of blocks written to the device in a single write job. If possible, will try to make a single pwrite call (default is 16, minimum is 1, maximum is 64)

MAX_BLOCKS_PER_HDD_READ_JOB
    maximum number of blocks read from the device in a single read job. If possible, will try to make a single pread call (default is 16, minimum is 1)

MAX_PARALLEL_HDD_READ_JOBS_PER_CS_ENTRY
    maximum number of read jobs processed in parallel per client connection (default is 1)

MAX_BUFFERS_POOL_SIZE_MB
    maximum size of the buffers pool used for read, write and replicate operations, in megabytes. Consider decreasing it if the system is running out of memory, or increasing it to improve performance if there is enough memory available. Unused buffers are regularly freed (default is 512)

READ_AHEAD_KB
    additional number of kilobytes which should be passed to posix_fadvise(POSIX_FADV_WILLNEED) before reading data from a chunk (default is 0, i.e. use posix_fadvise only with the amount of data that is really needed; the value is aligned down to 64 KiB)

MAX_READ_BEHIND_KB
    try to fix out-of-order read requests; the value tells how much of skipped data to read if an offset of some read operation is greater than the offset where the previous operation finished (default is 0, i.e. don’t read any skipped data; the value is aligned down to 64 KiB)

PERFORM_FSYNC
    call fsync() after a chunk is modified (default is 1, i.e. enabled)

STAT_CHUNKS_AT_DISK_SCAN
    verify that chunk metadata and data parts exists at scan time. If set to 1, the data part is also checked to be multiple of SFSBLOCKSIZE. Disabling this option accelerates the scan process, but does not check the chunk files. Warning: use it with caution, only in scenarios where a fast chunkserver restart is needed.(default is 1, i.e. enabled)

REPLICATION_TOTAL_TIMEOUT_MS
    total timeout for single replication operation. Replications that take longer than that are considered failed and are immediately aborted (default: 60000)

REPLICATION_CONNECTION_TIMEOUT_MS
    connection for single replication operation. If connecting to another chunkserver takes longer than this timeout, the operation is considered failed and is immediately aborted (default: 1000)

REPLICATION_WAVE_TIMEOUT_MS
    ADVANCED:: timeout for single wave in replication. After this timeout, next wave of read requests is sent to other chunkservers (default: 500)

PLUGINS_DIR
    directory to search for Chunkserver plugins (absolute path). When set, the plugin manager will search this directory first. If not set, then it will search in usual plugin locations, like PLUGINS_PATH/chunkserver (PLUGINS_PATH is defined at build time), /usr/local/lib/saunafs/plugins/chunkserver, and /usr/lib/saunafs/plugins/chunkserver (default is empty string, i.e. disabled)

CHUNK_TRASH_ENABLED (EXPERIMENTAL)
    enables or disables the chunk trash feature. When enabled, deleted chunks are moved to a trash directory instead of being immediately removed. (Default: 0)

CHUNK_TRASH_EXPIRATION_SECONDS (EXPERIMENTAL)
    specifies the timeout in seconds for chunks to remain in the trash before being permanently deleted. (Default: 259200)

CHUNK_TRASH_FREE_SPACE_THRESHOLD_GB (EXPERIMENTAL)
    sets the available space threshold in gigabytes. If the available space on the disk falls below this threshold, the system will start deleting older chunks from the trash to free up space. A suggested value is about the 15% of the capacity of the smaller configured disk. (Default: 0)

CHUNK_TRASH_GC_BATCH_SIZE (EXPERIMENTAL)
    defines the bulk size for the garbage collector when processing chunks in the trash. This determines how many files are processed in each garbage collection cycle. (Default: 1000)

CHUNK_TRASH_GC_SPACE_RECOVERY_BATCH_SIZE (EXPERIMENTAL)
    [ADVANCED] The number of files to remove from the trash in a single GC cycle, in case the disk is full and space needs to be recovered. (Default: 100)

LOG_LEVEL
    Setup logging. Uses the environment variable SAUNAFS_LOG_LEVEL or config value LOG_LEVEL to determine logging level. Valid log levels are

        • trace

        • debug

        • info

        • warn or warning

        • err or error

        • crit or critical

        • off

IO_PRIORITY_MODE (EXPERIMENTAL)
    Define IO jobs, i.e reads and writes from drives, prioritization scheme. The available modes are:

        • FIFO: prioritizes the IO jobs first enqueued.

        • SWITCH: alternates read and write jobs (interleaving) to prevent one class of IO from starving the other. Not reloadable. (Default: FIFO)

TLS_CERT_FILE (EXPERIMENTAL)
    Path to the TLS certificate file the chunk server will use for TLS connections (there is no default value).

TLS_KEY_FILE (EXPERIMENTAL)
    Path to the TLS private key file the chunk server will use for TLS connections (there is no default value).

TLS_CA_CERT_FILE (EXPERIMENTAL)
    Path to the trusted CA certificate which is used to authenticate through TLS to the master server (there is no default value).

REPORTING BUGS

Report bugs to the GitHub repository as an issue.

COPYRIGHT

Copyright 2008-2009 Gemius SA

Copyright 2013-2019 Skytechnology sp. z o.o.

Copyright 2023-2024 Leil Storage OÜ

SaunaFS is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, version 3.

SaunaFS is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.

You should have received a copy of the GNU General Public License along with SaunaFS. If not, see

SEE ALSO

sfschunkserver(8), sfshdd.cfg(5)