Configuration¶
Syntax¶
The Tron configuration file uses YAML syntax. The recommended configuration style requires only strings, decimal values, lists, and dictionaries: the subset of YAML that can be losslessly transformed into JSON. (In fact, your configuration can be entirely JSON, since YAML is mostly a strict superset of JSON.)
Past versions of Tron used additional YAML-specific features such as tags, anchors, and aliases. These features still work in version 0.3, but are now deprecated.
Basic Example¶
ssh_options:
agent: true
nodes:
- name: local
hostname: 'localhost'
jobs:
"getting_node_info":
node: local
schedule: "cron */10 * * * *"
actions:
"uname":
command: "uname -a"
"cpu_info":
command: "cat /proc/cpuinfo"
requires: [uname]
Command Context Variables¶
command attribute values may contain command context variables that are inserted at runtime. The command context is populated both by Tron (see Built-In Command Context Variables) and by the config file (see Command Context). For example:
jobs:
"command_context_demo":
node: local
schedule: "1st monday in june"
actions:
"print_run_id":
# prints 'command_context_demo.1' on the first run,
# 'command_context_demo.2' on the second, etc.
command: "echo {runid}"
SSH¶
- ssh_options (optional)
Options for SSH connections to Tron nodes. When tron runs a job on a node, it will add some jitter (random delay) to the run, which can be configured with the options below.
- agent (optional, default
False
) Set to
True
if trond should use an SSH agent. This requires that$SSH_AUTH_SOCK
exists in the environment and points to the correct socket.- identities (optional, default
[]
) List of paths to SSH identity files
- known_hosts_file (optional, default
None
) The path to an ssh known hosts file
- connect_timeout (optional, default
30
) Timeout in seconds when establishing an ssh connection
- idle_connection_timeout (optional, default
3600
) Timeout in seconds that an ssh connection can remain idle after which it is closed
- jitter_min_load (optional, default
4
) Minimum load on a node before any jitter is introduced. See jitter_load_factor for a description of how load is calculated
- jitter_max_delay (optional, default
20
) Maximum number of seconds to add to a run
- jitter_load_factor (optional, default
1
) Factor used to increment the count of running actions for determining the upper bound of jitter to add (ex. A factor of 2 would increase the upper bound by 2 seconds per running action)
- agent (optional, default
Example:
ssh_options:
agent: false
known_hosts_file: /etc/ssh/known_hosts
identities:
- /home/batch/.ssh/id_dsa-nopasswd
connect_timeout: 30
idle_connection_timeout: 3600
jitter_min_load: 4
jitter_max_delay: 20
jitter_load_factor: 1
Time Zone¶
- time_zone (optional)
Local time as observed by the system clock. If your system is obeying a time zone with daylight savings time, then some of your jobs may run early or late on the days bordering each mode. See Notes on Daylight Saving Time for more information.
Example:
time_zone: US/Pacific
Command Context¶
- command_context
Dictionary of custom command context variables. It is an arbitrary set of key-value pairs.
Example:
command_context:
PYTHON: /usr/bin/python
TMPDIR: /tmp
See a list of Built-In Command Context Variables.
Output Stream Directory¶
- output_stream_dir
A path to the directory used to store the stdout/stderr logs from jobs. It defaults to the
--working_dir
option passed to trond.
Example:
output_stream_dir: "/home/tronuser/output/"
State Persistence¶
- state_persistence
Configure how trond should persist its state to disk. By default a shelve store is used and saved to ./tron_state in the working directory.
- store_type
- Valid options are:
shelve - uses the shelve module and saves to a local file
yaml - uses yaml and saves to a local file (this is not recommend and is provided to be backwards compatible with previous versions of Tron).
You will need the appropriate python module for the option you choose.
- name
The name of this store. This will be the filename for a shelve or yaml store.
- buffer_size
The number of save calls to buffer before writing the state. Defaults to 1, which is no buffering.
Example:
state_persistence:
store_type: shelve
name: tron_store
Action Runners¶
Note: this is an experimental feature
- action_runner
Action runner configuration allows you to run Job actions through a script which records it’s pid. This provides support for a max_runtime option on jobs, and allows you to stop or kill the action from tronctl.
- runner_type
- Valid options are:
- none
Run actions without a wrapper. This is the default
- subprocess
Run actions with a script which records the pid and runs the action command in a subprocess (on the remote node). This requires that bin/action_runner.py and bin/action_status.py are available on the remote host.
- remote_status_path
Path used to store status files. Defaults to /tmp.
- remote_exec_path
Directory path which contains action_runner.py and action_status.py scripts.
Example:
action_runner:
runner_type: "subprocess"
remote_status_path: "/tmp/tron"
remote_exec_path: "/usr/local/bin"
Nodes¶
- nodes
List of nodes. Each node has the following options:
- hostname (required)
The hostname or IP address of the node
- name (optional, defaults to
hostname
) A name to refer to this node
- username (optional, defaults to current user)
The name of the user to connect with
- port (optional, defaults to 22)
The port number of the node
Example:
nodes:
- name: node1
hostname: 'batch1'
- hostname: 'batch2' # name is 'batch2'
Node Pools¶
- node_pools
List of node pools, each with a
name
andnodes
list.name
defaults to the names of each node joined by underscores.
Example:
node_pools:
- name: pool
nodes: [node1, batch1]
- nodes: [batch1, node1] # name is 'batch1_node1'
Jobs and Actions¶
- jobs
List of jobs for Tron to manage. See Jobs for the options available to jobs and their actions.
Logging¶
As of v0.3.3 Logging is no longer configured in the tron configuration file.
Tron uses Python’s standard logging and by default uses a rotating log file
handler that rotates files each day. The default log directory is
/var/log/tron/tron.log
.
To configure logging pass -l <logging.conf> to trond. You can modify the default logging.conf by copying it from tron/logging.conf. See http://docs.python.org/howto/logging.html#configuring-logging
Interesting logs¶
Most tron logs are named by using pythons __file__ which uses the modules name. There are a couple special cases:
- twisted
Twisted sends its logs to the twisted log
- tron.api.www.access
API access logs are sent to this log at the INFO log level. They follow a standard apache combined log format.