Frontier Client Usage
Building C/C++ Client Libraries and Test Executables
- The latest Frontier client source code can be downloaded from http://frontier.cern.ch/dist/.
- The distribution contains everything necessary to build C/C++ Frontier
clients, including a recent version of expat libraries.
- To build the client libraries and test executables one can simply execute
make in the client source directory.
- These make command line variables can be defined to select non-default
libraries for support packages:
-
EXPAT_DIR - expat (overrides included library)
-
ZLIB_DIR - zlib (overrides system library)
-
OPENSSL_DIR - openssl (overrides system library)
-
PACPARSER_DIR - pacparser (default is to look in system but it is not a
standard Linux package in Redhat).
- To prepare source code distribution file: make src-dist (will create tar
file in the client/tmp directory)
- To prepare library distribution file: make dist (will create tar file in
the client/tmp directory)
- Client release notes have been maintained in the file RELEASE_NOTES.
Frontier client configuration can can come from a complex server
connection string made up of multiple keyword=value pairs enclosed in
parentheses. For example:
(serverurl=http://cmsfrontier.cern.ch:8000)(retrieve-ziplevel=0)...
If there are any parentheses in a server connection string, anything
outside of parentheses is ignored (for example, it is often convenient
to precede the string with http:// to pretend the whole string is a
server url). If any of the configuration items are missing from the
server connection string, they can also be read from environment
variables. These are the connection string keywords and corresponding
environment variables when applicable:
-
serverurl - server of the form http://host:port/servlet (:port is
optional and defaults to 80). Multiple serverurl keywords may be
supplied and they will be tried in the order listed. If a host
DNS name contains multiple IP addresses, normally only one of those
addresses is tried because the server is contacted by a proxy;
however, when connecting directly to a server and the first addresses
tried fail, all addresses will be tried. The full retry strategy
is documented
here.
Environment
variables FRONTIER_SERVER and FRONTIER_SERVER[123456] can add
additional servers. Each server string also may contain any
parenthesized keywords. The maximum number of servers is 16.
-
proxyurl - proxy of the form http://host:port. Multiple proxyurl
keywords may be supplied and they will be tried in the order listed.
When a DNS lookup on the host returns multiple IP addresses and the
first ones tried fail, all values will be tried.
Environment variables FRONTIER_PROXY and FRONTIER_PROXY[1234] can add
additional proxies (or parenthesized keywords in FRONTIER_SERVER like
all the other keywords), up to a maximum of 24 total proxies. See also
backupproxyurl below.
-
backupproxyurl - Just like proxyurl, except that they are always tried
after all proxyurls and they are not included in the load balancing
done when loadbalance=proxies. Implies failovertoserver=no.
-
proxyconfigurl - URL pointing to a
Proxy Auto-Config
(PAC) file that can return a list of proxies. May begin with "http://"
or "file://", or "auto" which is replaced with "http://wpad/wpad.dat".
The first serverurl listed is given as the url/host parameter to the
FindProxyForURL function in the file. Multiple proxyconfigurl
keywords may be used so if http requests fail the next one will be
tried, but if all fail it is a fatal error. If a proxyconfigurl file:// request
fails it is immediately a fatal error (that is, any further
proxyconfigurl keywords will be ignored). Another difference with
file:// is that a less reliable method is used for finding the
client's IP address for use inside of the PAC file in the
myIpAddress() function (using a DNS lookup on the hostname as opposed
to finding the outgoing IP address used to connect to the http
server). If proxyurl keywords are also specified, they will be
considered lower priority than the proxies returned from proxyconfigurl;
however, client versions older than version 2.8.20 made proxyurls higher
priority, so to be backward compatible with older clients, it is
safest to list proxies with backupproxyurl instead of proxyurl.
PAC URLs may also be semicolon-separated in a
FRONTIER_PROXYCONFIGS environment variable.
-
logicalserverurl - logical server of the form http://logical_name
which, when encountered in a later connection as a server name, will
be replaced by the entire rest of the complex connection string that
the logicalserverurl keyword came from. If logicalserverurl is not
present in the connection string, it may come from environment variable
FRONTIER_LOGICALSERVER and then its replacement string comes from
FRONTIER_PHYSICALSERVERS.
-
retrieve-ziplevel - zip level of retrieved data, from 0 for none to
1 for fast to 9 for best. If not present, may come from environment
variable FRONTIER_RETRIEVEZIPLEVEL. Default 5.
-
connecttimeoutsecs - number of seconds before timeout on a server or
proxy connection. If not present, may come from environment variable
FRONTIER_CONNECTTIMEOUTSECS. Default 5.
-
readtimeoutsecs - number of seconds before timeout on a read from a
server or proxy. If not present, may come from environment variable
FRONTIER_READTIMEOUTSECS. Default 10.
-
writetimeoutsecs - number of seconds before timeout on a write to a
server or proxy. If not present, may come from environment variable
FRONTIER_WRITETIMEOUTSECS. Default 5.
-
forcereload - value of "short" forces an immediate cache refresh
on all short time-to-live queries (those indicated by setReload API),
and a value of "long" forces an immediate refresh on all queries.
If value begins with "soft", do a softer refresh request which just
asks that the caches be re-validated and so is a bit gentler on
the servers. If not present, may come from environment variable
FRONTIER_FORCERELOAD. Default "none".
-
maxagesecs - maximum number of seconds that the client allows for
caching. The default is to use the number of seconds specified by the
host. Note that "maxagesecs=0" is equivalent to "forcereload=softlong".
-
freshkey - anything in the value is added to the URLs used for queries,
so that if a new value is used the queries will be guaranteed to not
be in the cache. If the value begins with "long" that is removed and
the rest of the value is added to all queries, otherwise the value is
added to only "short" time-to-live queries. If not present, may come
from environment variable FRONTIER_FRESHKEY. Default is no extra key.
-
loadbalance - value of 'proxies' or 'servers' (both may be listed
with loadbalance=proxies and loadbalance=servers) to cause frontier
client to do its own loadbalancing between the listed proxies and
or servers. If proxyconfigurl is also used, then loadbalance=proxies
is ignored.
-
clientcachemaxresultsize - maximum result size (in bytes) that will
be cached in a local cache inside the frontier client. Default 0.
Identical queries whose results are cached locally will not
go to a proxy or server to retrieve the answer.
-
failovertoserver - when set to a value of 'no', prevents failing over
to direct connections to servers if proxy connections fail. Default 'yes'.
-
preferipfamily - controls the ordering of IP addresses tried when both
IPv4 and IPv6 values are present in a DNS lookup. When set to "6",
prefers IPv6. When set to "4", prefers IPv4. When set to "0", uses
the order set by the operating system. Default is "4". When attempts
fail, all values are tried; this only controls the order.
-
security - When set to "sig", requests that the server send digital
signatures on every response. Requires access to a directory of
Certifying Authority certificates and CRLs.
-
capath - Directory containing Certifying Authority certificates and CRLs.
If not present, may come from environment variable X509_CERT_DIR.
Default is /etc/grid-security/certificates.
The following may only come from environment variables:
-
FRONTIER_ID - an identifying string that will be put into the squid
and frontier server logs
-
FRONTIER_LOG_LEVEL - set logging level
-
nolog - no log output (default)
-
error - log errors
-
warning - log errors and warnings (which mainly have to do with non-fatal
connection problems and retries)
-
debug - log errors, warnings, and verbose debug messages
-
-
FRONTIER_LOG_FILE - filename to write log messages to instead of stdout.
If the filename begins with "+", and $FRONTIER_LOG_LEVEL is set to warning
or debug, the "+" is removed from the log file name and warning messages
get sent to stdout in addition to the log file. If there's a '%P' in
the filename, it is replaced with the current process id.
Note that isn't part of frontier client but may be useful information
to some readers: in CMS, a long connection string is constructed from
the frontier-connect portion of site-local-config.xml in
$CMS_PATH/SITECONF/local/JobConfig. Blanks and quotes are removed
from that and they're converted into the parenthesized form above,
plus a (logicalserverurl=http://cms_conditions_data) is added so that
form can be used a shortcut.
C++ Test Client Usage
Writing C++ Clients
- The C++ classes which handle communication with Frontier servers
and proxies are the Connection and Session classes.
A Connection object can be used to hold a connection open for multiple
queries, and queries are done one per Session object. Here is an
example of creating an instance of each:
#include
#include
#include "frontier_client/frontier-cpp.h"
...
std::list serverList;
serverList.push_back("http://frontier.cern.ch:8000/FrontierInt");
std::list proxyList;
frontier::Connection connection(serverList, proxyList);
frontier::Session session(&connection);
...
Note that the Connection object created via the default class
constructor will be configured using the above mentioned environment
variables (FRONTIER_SERVER[1,2,3,4], FRONTIER_PROXY[1,2,3,4]).
- Frontier requests are handled via the Request class:
...
std::string sql("select 1 from dual");
std::string parameter1 = frontier::Request::encodeParam(sql);
frontier::Request request("FrontierTestRequest:1", frontier::BLOB);
request.addKey("p1", parameter1);
std::vector requestVector;
requestVector.push_back(&request);
- The Session object retrieves request data from the Frontier
server, and provides an iterator for going over the returned records:
// Retrieve data.
session.getData(requestVector);
session.setCurrentLoad(1);
// Go over the object fields (metaData consists of one record with field names).
int field_num=0;
std::cout << "Object fields:" << std::endl;
std::string name, type;
session.next();
while(!session.isEOR()) {
session.assignString(&name);
session.assignString(&type);
++field_num;
std::cout << field_num << " " << (name) << " " << (type) << std::endl;
}
// Go over the actual records and extract them according to their type.
std::cout << " \nResult contains "<< session.getNumberOfRecords() <<" objects.\n";
frontier::AnyData ad;
std::string stringBuf;
std::string *vs = &stringBuf;
while(session.next()) {
for(int k=0;k<field_num;k++) {
session.getAnyData(&ad);
switch(ad.type()) {
case frontier::BLOB_TYPE_INT4:
int vi = ad.getInt();
std::cout << vi;
break;
case frontier::BLOB_TYPE_INT8:
long long vl = ad.getLongLong();
std::cout << vl;
break;
case frontier::BLOB_TYPE_FLOAT:
float vf = ad.getFloat();
std::cout << vf;
break;
case frontier::BLOB_TYPE_DOUBLE:
double vd = ad.getDouble();
std::cout << vd;
break;
case frontier::BLOB_TYPE_TIME:
long long vl=ad.getLongLong();
std::cout << vl;
break;
case frontier::BLOB_TYPE_ARRAY_BYTE:
ad.assignString(vs);
if(vs->size() != 0) {
std::cout << "NULL";
}
else {
str_escape_quota(vs);
std::cout << '\'' << (*vs) << '\'';
}
break;
default:
std::cout << "Error: unknown typeId " << ((int)(ad.type())) << "\n";
exit(1);
}
}
}
- All exceptions that can be thrown from the Frontier libraries
inherit from the FrontierException class. Therefore, one
should be able to enclose all Frontier calls into try/catch block as
follows:
try {
// Construct frontier objects and do something.
}
catch(const frontier::FrontierException& e) {
std::cout << "Frontier exception caught: " << e.what() << std::endl;
// Handle generic error.
}
All frontier exceptions (e.g., ConfigurationError,
InvalidArgument, NetworkProblem, etc.) are listed
in the include/frontier_client/FrontierException.hpp file.
- A working example of a frontier C++ client can be found in the file
test-req.cc in the source code distribution.
Python Client Usage
Dave Dykstra
Last modified: 8 Feb 2017