Depot
From DAMSL Wiki
Contents |
Installation
Download Phoebus
- Nightly snapshots may be downloaded here phoebus_nightly.tar.gz
- Phoebus source may also be downloaded directly from the SVN repository:
svn co https://damsl.cis.udel.edu/svn/lsl/trunk
Build instructions
- A basic installation can be performed as follows on most platforms:
# ./autogen.sh # ./configure # make && make install
- Some common configure flags
--enable-debug ### Enable debugging output for both client wrapper and Phoebus daemon --enable-netlogger ### Compile with NetLogger features
Configuration
Syntax
The Phoebus depot configuration file is a hierarchical variant of the common name/value pairs configuration file.
All elements would be of the form:
name = value;
The value field can be an integer, floating-point number, boolean or string depending on the variable.
boolean example: use_tracerate=TRUE; string example: route_file="/home/aaron/route.dat"; integer example: incoming_port=5006; floating point example: incoming_port=50.07;
Elements can be grouped like so:
logger {
backend="file";
output_file="/tmp/psd.out";
}
Groups can also appear inside of groups:
outgoing {
default {
tcp {
use_web100 = FALSE;
ipv6_enabled=TRUE;
}
}
}
Comments can be written as any content coming after a # symbol. This can appear anywhere on a line, and all subsequent characters will be ignored.
Settings
The Phoebus settings can be broadly divided into two sections: general depot settings and per connection settings. The configuration file divides the settings into two top-level groups: depot and connections.
Depot settings consist of the general settings, like logging and modules, that apply no matter what the connection is.
Per connection settings are the protocol settings and similar set for each connection.
Depot Settings
Listening For Incoming Connections
The listeners group is used to specify with the protocol options to use when listening for incoming connections. These options include things like socket buffersizes, timeouts, and ports.
On startup, the daemon will go through and start listeners for each protocol loaded. If no protocol settings are specified for a given protocol, the defaults will be used. To disable a given protocol, create a group element for it and set the 'disabled' option to TRUE.
An example listeners block is shown below.
listeners {
tcp {
port = 5008; # Set the port to listen on
recv_bufsize = 16777216; # Set the buffer size to use for incoming connections.
send_bufsize = 16777216;
}
udt {
disabled = TRUE; # Disable this protocol
port = 5008; # Set the port to listen for UDT connections on
}
}
Modules
Many of the protocols and other features of the depot are implemented as modules. By default, only the TCP and anonymous authentication modules will be loaded.
All module options are set in a top-level group called 'module'.
Module options:
| dir | A string specifying the directory to load the modules from |
| list | A string containing modules to load separated by a space |
Example which will load the TCP, Anonymous Authentication and UDT modules from /usr/lib/phoebus:
module {
dir="/usr/lib/phoebus/";
list="tcp auth_anon udt";
}
Available modules:
| tcp | Enable incoming/outgoing TCP connections |
| udt | Enable incoming/outgoing UDT connections |
| oscars_basic | Allows allocation of OSCARS circuits before making outgoing connections |
| dragon_basic | Allows allocation of DRAGON circuits before making outgoing connections |
| auth_pass | Enable password authentication |
| auth_anon | Enable anonymous authentication |
| lsink | Simple modules that sends a file. Useful mainly for debugging. Requires using the included lput program. |
| lsrc | Simple modules that receives a file. Useful mainly for debugging. Requires using the included lget program. |
Logging
All logging variables will appear in a top-level group named 'logger':
The `backend` variable specifies where the output will go to. Can be one of `stdout` or `file`.
To log to the screen (which is the default):
logger {
backend="stdout";
}
To log to a file, you would do something like this:
logger {
backend="file";
output_file="/path/to/file.out";
}
Routing
Routing is used to choose the next hop for a given destination. Currently, there are two options available static routes and traceroute.
Static Routes
Static routes allow you to assign a depot to a specific subnet or host. If a request comes in with a destination in that subnet, the depot will use the assigned depot as the next hop instead of directly connecting to the destination.
To configure this, you need to specify the file containg the list of subnet/depot pairs:
Example psd.conf:
route {
route_file="/path/to/route.dat";
}
The route_file is the file containing the routing information, it takes the form of subnets and depot ids. The subnet and depot ids are tab separated (by as many tabs as desired).
Example route.dat:
140.232.0.0/16 140.232.101.101/5008 10.0.0.0/24 10.0.0.5/5006
In this file, any request coming in with a destination in the 140.232.0.0/16 subnet will be sent to the depot at 140.232.101.101/5008. If a request comes in to a host in the 10.0.0.0/24 subnet, it will be sent to the 10.0.0.5/5006 depot.
To verify that this is properly enabled, you should see messages like the following on startup:
2008/04/09 12:09:52 INFO(8): __psd_route_list(): 64.57.17.134/32 -> 10.0.0.65/5006
Traceroute Routing
Traceroute routing uses traceroute to find depots along the path between the current depot and the destination. To enable it, you must specify the "use_traceroute" boolean and a file containing route information(specified with the "traceroute_file" variable). Static routing and traceroute routing can be used in conjunction. If no static route is found, a dynamic one will be tried.
Example phoebus.conf:
route {
use_traceroute=TRUE;
traceroute_file="/path/to/traceroute.dat".
}
The traceroute file consists of IPs or hostnames and the depots. The IPs/hostnames correspond to interfaces that might be seen along the path between a source and destination. For example, if one were to setup a host in los angeles and knew that all traceroutes between hosts outside los angeles to hosts inside los angeles would pass through a certain router or set of routers, these could be specified and the along with the los ageneles depot to route all traffic destined for los angeles through that depot. If multiple options are seen along the path, the one furthest along is chosen. The router hostname/ip and the depot id are specified by as many tabs as desired.
Example traceroute.dat:
so-3-0-0.0.rtr.losa.net.internet2.edu 10.0.0.31/5008 so-0-1-0.0.rtr.hous.net.internet2.edu 10.0.0.41/5008 172.23.15.23 10.0.0.51/5008
With this file, if a traceroute to the destination host sees so-3-0-0.0.rtr.losa.net.internet2.edu, the depot will use the 10.0.0.31/5008 depot as the next hop. If so-0-1-0.0.rtr.hous.net.internet2.edu is seen, it will use the 10.0.0.41/5008 depot as the next hop.
To verify that this is properly enabled, you should see lines similar to the following on startup:
2008/03/05 17:44:54 INFO(0): psd_traceroute_file_read(): added h "so-3-1-0.0.rtr.salt.net.internet2.edu"->"10.0.0.21/5006"
Connection Settings
All connection settings are contained in the "connections" group.
* Configuring TCP options * Configuring UDT options * Configuring Circuit options
Example traceroute.dat
so-0-0-0.0.rtr.salt.net.internet2.edu 10.0.0.21/5006 so-3-0-0.0.rtr.losa.net.internet2.edu 10.0.0.31/5006 so-0-1-0.0.rtr.hous.net.internet2.edu 10.0.0.41/5006
Example route.dat
0.0.0.0/0 10.0.0.21/5006 199.109.33.61/32 direct 193.1.228.67/32 10.0.0.1/5006 64.57.17.134/32 10.0.0.31/5006
Example psd.conf
# The depot group is where depot specific options are configured.
depot {
# modules {
# list="auth_anon tcp oscars_basic"; # Which modules should be loaded?
# dir="/usr/lib/psd/"; # Where should the modules be loaded from?
# }
user="phoebus"; # what user should the daemon run as?
group="phoebus"; # what group should the daemon run as?
pid_file="/var/run/phoebus.pid"; # What file the daemon should write its PID to
logger {
backend="file"; # Where should the logging go? stdout, file or syslog
output_file="/var/log/phoebus.log"; # Which file should logging go to?
}
# The route group is used to configure routing options in the daemon.
# This includes which files to find the routing.
#
# route {
# route_file="/etc/phoebus/route.dat"; # The file containing subnet/depot mappings
# use_traceroute=TRUE; # Should traceroute routing be used?
# traceroute_file="/etc/phoebus/traceroute.dat"; # A file containing IP/depot mappings
# traceroute_max_ttl=15; # How many hops should it search through?
# traceroute_max_timeout=2; # How long should traceroute wait between hops?
# }
#
# The listeners group is used to setup the ports on which to listen for
# incoming connection requests.
#
# listeners {
# udt {
# port=5006;
# recv_bufsize=0;
# send_bufsize=0;
# }
#
# tcp {
# port = 5006;
# ipv6_enabled=TRUE;
# ipv4_enabled=TRUE;
# recv_timeout=0;
# send_timeout=0;
# recv_bufsize=0;
# send_bufsize=0;
# }
# }
}
# The connections group holds the per connection settings
# - It has 3 groups in it: default, incoming and outgoing.
# - Incoming is currently unused
# - Default settings apply to all connections (incoming or outgoing)
# - Outgoing settings are applied to each outgoing connection and can override any of the default settings.
# connections {
# default {
# # These oscars settings apply to all outgoing connections. This
# # approach is used to keep us from having to repeat these
# # elements in every set of OSCARS settings.
# oscars {
# java_path = "/opt/jdk1.6.0_03/bin/java"; # The path to the java binary. if unset, it will use the java in the path
# axis_path = "/opt/axis2-1.3"; # The path to the directory containing axis2 and rampart
# client_directory = "/opt/oscars_client"; # The path to the directory containing the OSCARS client
# server = "https://server.edu:8443/services/OSCARS"; # The URL of the IDC to request circuits with
# duration = 1200; # The amount of time to request during the circuit allocation
# sleep_time = 10; # How long to pause after the circuit is returned as active before connecting.
# teardown_timeout = 90; # How long to wait after the last connection using a circuit is down before tearing the circuit down.
# }
# }
#
# # Inside outgoing, there are two types of groups: default and route.
# # The default group can be used to configure settings that will apply to
# # all outgoing connections. The route groups can be used to apply
# # settings applicable to a subnet or single host. If settings exist for
# # both a host and the subnet it's in, the settings from the host will be
# # used in place of those for the subnet.
# outgoing {
# default {
# tcp {
# use_web100 = FALSE; # Used to selectively enable or disable web100 monitoring
# ipv6_enabled=TRUE; # Used to selectively enable to disable IPv6 connections
# ipv4_enabled=TRUE; # Used to selectively enable to disable IPv4 connections
# recv_timeout=0; # Set the receive timeout on connections. If 0, it's the default.
# send_timeout=0; # Set the send timeout on connections. If 0, it's the default.
# recv_bufsize=0; # Set the send buffer size on connections. If 0, it's the default.
# send_bufsize=0; # Set the receive buffer size on connections. If 0, it's the default.
# }
# }
#
# # Settings for connecting to any host in the given subnet:
# route:10.0.1.0/24 {
# bandwidth = 1000;
# protocols = [ "TCP" ];
#
# tcp {
# recv_bufsize=16777216;
# send_bufsize=16777216;
# }
# }
#
# # Settings for connecting to a specific depot:
# route:10.0.2.41/32 {
# bandwidth = 1000;
# protocols = [ "UDT" ]; # Use UDT to connect to the host
# path_protocols = [ "OSCARS" ]; # Use OSCARS to allocate a circuit
#
# oscars {
# src_id = "blackie.nysernet.org"; # The OSCARS source name or identifier
# dst_id = "phoebus.hous.dcn.internet2.edu"; # The OSCARS destination name or identifier. Order doesn't really matter since the circuit is bidirectional.
# type = "shared"; # If you want to allow only one Phoebus connection to use this circuit, set the type to "private".
# }
# }
# }
#}
