Traffic Ops
At one point, Traffic Ops was a collection of Perl scripts, and while the current program is written in Go, many of its tools and utilities are still written in Perl.
Introduction
Traffic Ops at its core is mainly a PostgreSQL database used to store configuration information for ATC, and a set of RESTful API endpoints for interacting with and manipulating that information. It also serves as the single point of authentication for ATC components (that is, when one hears “user” in an ATC context it nearly always means a “user” as configured in Traffic Ops) and provides interfaces to other ATC components by proxy. Additionally, there is some miscellaneous, at times obscure functionality to Traffic Ops, such as generating arbitrary Linux system images.
Software Requirements
Traffic Ops is only supported on CentOS 7+ systems (although many developers do use Mac OS with some success). Here are the requirements:
PostgreSQL 13.2 - the machine where Traffic Ops is running must have the client tool set (e.g. psql(1)), but the actual database can be run anywhere so long as it is accessible.
Note
Prior to version 13.2, Traffic Ops used version 9.6. For upgrading an existing Mac OS Homebrew-based PostgreSQL instance, you can use Homebrew to easily upgrade from 9.6 to 13.2:
brew services stop postgresql brew upgrade postgresql brew postgresql-upgrade-database brew cleanup postgresql@9.6 brew services start postgresql
openssl(1SSL) is recommended to generate server certificates, though not strictly required if certificates can be obtained by other means.
Some kind of SMTP server is required for certain Traffic Ops API endpoints to work, but for purposes unrelated to them an SMTP server is not required. CDN in a Box comes with a relayless SMTP server for testing (you can view the emails that Traffic Ops sends, but they aren’t sent anywhere outside CDN-in-a-Box).
Tip
Alternatively, development and testing can be done using CDN in a Box - albeit somewhat more slowly.
Perl Script Requirements
Not much code is still in Perl, but for the scripts the following are needed:
Perl 5.10.1
libpcap and libpcap development library - usually
libpcap-dev
orlibpcap-devel
in your system’s native package manager.libpq and libpq development library - usually
libpq-dev
orlibpq-devel
in your system’s native package manager.Developers should use Perltidy to format their Perl code.
-l=156 -et=4 -t -ci=4 -st -se -vt=0 -cti=0 -pt=1 -bt=1 -sbt=1 -bbt=1 -nsfs -nolq -otr -aws -wls="= + - / * ." -wrs=\"= + - / * .\" -wbb="% + - * / x != == >= <= =~ < > | & **= += *= &= <<= &&= -= /= |= + >>= ||= .= %= ^= x="
Go Program Requirements
If the system’s Go compiler doesn’t provide it implicitly, also note that all Go code in the ATC repository should be formatted using gofmt
All Go code dependencies are managed through the go.mod
, go.sum
, and vendor/modules.txt
files. With the exception of golang.org/x
packages (see below), module dependencies in vendor/
are tracked in Git and should thus be available without any extra work - and any new dependencies should be properly “vendored” into that same, top-level directory. No other vendor/
directories exist, as Go modules only supports a single vendor directory.
Per the Go language standard’s authoritative source’s recommendation, all sub-packages of golang.org/x
are treated as a part of the compiler, and so need not ever be “vendored” as though they were an external dependency. These dependencies are not listed explicitly here, so it is strongly advised that they be fetched using go-get(1) rather than downloaded by hand.
Tip
All new dependencies need to be subject to community review to ensure necessity (because it will be added in its entirety to the repository, after all) and license compliance via the developer mailing list.
Traffic Ops Project Tree Overview
traffic_ops/
- The root of the Traffic Ops projectapp/ - Holds most of the Perl code base, though many of the files contained herein are also used by the Go implementation
Note
This directory is home to many things that no longer work as intended or have been superseded by other things.
bin/ - Directory for scripts and tools, cron(8) jobs, etc.
checks/ - Contains the Check Extensions scripts that are provided by default
db/ - Contains scripts that manipulate the database beyond the scope of setup, migration, and seeding
tests/ - Integration and unit test scripts for automation purposes - in general this has been superseded by
traffic_ops/testing/api/
conf/ - Aggregated configuration for Traffic Ops. For convenience, different environments for the app/db/admin tool are already set up
development/ - Configuration files for the “development” environment
integration/ - Configuration files for the “integration” environment
misc/ - Miscellaneous configuration files.
production/ - Configuration files for the “production” environment
test/ - Configuration files for the “test” environment
db/ - Database setup, seeding, and upgrade/downgrade helpers
migrations/ - Database migration files
tools/ - Contains helper scripts for easing upgrade transitions when selective data manipulation must be done to achieve a desirable state
script/ - Mojolicious bootstrap/startup scripts.
templates/ - Mojolicious Embedded Perl (
template name.ep
) files for the now-removed Traffic Ops UI
build/ - Contains files that are responsible for packaging Traffic Ops into an RPM file - and also for doing the same with ORT
etc/ - Configuration files for various systems associated with running production instances of Traffic Ops, which are installed under
/etc
by the Traffic Ops RPMcron.d/ - Holds specifications for cron(8) jobs that need to be run periodically on Traffic Ops servers
Note
At least one of these jobs expects itself to be run on a server that has the Perl implementation of Traffic Ops installed under
/opt/traffic_ops/
. Nothing terrible will happen if that’s not true, just that it/they won’t work. Installation using the RPM will set up all of these kinds of things up automatically.init.d/ - Contains the old, initscripts-based job control for Traffic Ops
logrotate.d/ - Specifications for the Linux logrotate(8) utility for Traffic Ops log files
profile.d/traffic_ops.sh - Sets up common environment variables for working with Traffic Ops
install/ - Contains all of the resources necessary for a full install of Traffic Ops
bin/ - Binaries related to installing Traffic Ops, as well as installing its prerequisites, certificates, and database
data/ - Contains things that need to be accessible by the running server for certain functionality - typically installed to
/var/www/data
by the RPM (hence the name).etc/ - This directory left empty; it’s used to contain post-installation extensions and resources
lib/ - Contains libraries used by the various installation binaries
ort/ - Contains ORT and ATS configuration file-generation logic and tooling
testing/ - Holds utilities for testing the Traffic Ops API
api/ - Integration testing for the Traffic Ops Go client (
github.com/apache/trafficcontrol/v8/traffic_ops/v4-client
) and Traffic Ops
traffic_ops_golang/ - The root of the Go implementation’s code-base
Note
The vast majority of subdirectories of
traffic_ops/traffic_ops_golang/
contain handlers for the Traffic Ops API, and are named according to the endpoint they handle. What follows is a list of subdirectories of interest that have a special role (i.e. don’t handle a Traffic Ops API endpoint).See also
The GoDoc documentation for this package:
github.com/apache/trafficcontrol/v8//
api/ - A library for use by Traffic Ops API handlers that provides helpful utilities for common tasks like obtaining a database transaction handle or accessing Traffic Ops configuration
auth/ - Contains definitions of privilege levels and access control code used in routing and provides a library for dealing with password and token-based authentication
config/ - Defines configuration structures and methods for reading them in from files
dbhelpers/ - Assorted utilities that provide functionality for common database tasks, e.g. “Get a user by email”
plugin/ - The Traffic Ops plugin system, with examples
trafficvault/ - This package provides the Traffic Vault interface and associated backend implementations for other handlers to interact with Traffic Vault.
routing/ - Contains logic for mapping all of the Traffic Ops API endpoints to their handlers, as well as proxying requests back to the Perl implementation and managing plugins, and also provides some wrappers around registered handlers that set common HTTP headers and connection options
swaggerdocs/ A currently abandoned attempt at defining the Traffic Ops API using Swagger - it may be picked up again at some point in the (distant) future
tenant/ - Contains utilities for dealing with Tenantable resources, particularly for checking for permissions
tocookie/ - Defines the method of generating the
mojolicious
cookie used by Traffic Ops for authenticationvendor/ - contains “vendored” Go packages from third party sources
v3-client - The official Traffic Ops Go client package for working with the version 3 Traffic Ops API.
v4-client - The official Traffic Ops Go client package for working with the version 4 Traffic Ops API.
v5-client - The official Traffic Ops Go client package for working with the version 5 Traffic Ops API.
vendor/ - contains “vendored” Go packages from third party sources
app/db/admin
The app/db/admin binary is for use in managing the Traffic Ops (and Traffic Vault PostgreSQL backend) database tables. This essentially serves as a front-end for Migrate, though dbconf.yml
comes from Goose, which Traffic Ops used to use before switching to Migrate.
Note
For proper resolution of configuration and SOL statement files, it’s recommended that this binary be run from the app
directory
Usage
db/admin [options] command
Options and Arguments
- --env ENVIRONMENT
An optional environment specification that causes the database configuration to be read out of the corresponding section of the
app/db/dbconf.yml
configuration file. One of:development
integration
production
test
admin sets
MOJO_MODE
to the value of the environment as specified by this option. (Default:development
)
- --trafficvault
When used, commands will be run against the Traffic Vault PostgreSQL backend database as specified in the
app/db/trafficvault/dbconf.yml
configuration file.
- command
The
command
specifies the operation to be performed on the database. It must be one of:- createdb
Creates the database for the current environment
- create_migration
Creates a pair of timestamped up/down migrations titled NAME.
- create_user
Creates the user defined for the current environment
- dbversion
Displays the database version that results from the current sequence of migrations
- down
Rolls back a single migration from the current version
- drop
Drops the database for the current environment
- drop_user
Drops the user defined for the current environment
- load_schema
Sets up the database for the current environment according to the SQL statements in
app/db/create_tables.sql
orapp/db/trafficvault/create_tables.sql
if the--trafficvault
option is used- migrate
Runs a migration on the database for the current environment
- patch
Patches the database for the current environment using the SQL statements from the
app/db/patches.sql
. This command is not supported when using the--trafficvault
option- redo
Rolls back the most recently applied migration, then run it again
- reset
Creates the user defined for the current environment, drops the database for the current environment, creates a new one, loads the schema into it, and runs a single migration on it
- seed
Executes the SQL statements from the
app/db/seeds.sql
file for loading static data. This command is not supported when using the--trafficvault
option- show_users
Displays a list of all users registered with the PostgreSQL server
- status
Deprecated,
status
is now an alias fordbversion
and will be removed in a future Traffic Control release.- upgrade
Performs a migration on the database for the current environment, then seeds it and patches it using the SQL statements from the
app/db/patches.sql
file
db/admin --env=test reset
The environments are defined in the traffic_ops/app/db/dbconf.yml
file, and the name of the database generated will be the name of the environment for which it was created. If the --trafficvault
option is used, the app/db/trafficvault/dbconf.yml
file defines this information.
Resolving Migration Failures
If you encounter an error running a migration, you will see a message like
[root@trafficops app]# db/admin -env production migrate
Error running migrate up: migration failed: syntax error at or near "This_is_a_syntax_error" (column 1) in line 18: /*
That means that the migration timestamp in the version
column of the schema_migrations
table has been updated to the version of the migration that failed, but the dirty
column is also set, and if you try to run another migration (either up or down), you will see
[root@trafficops app]# db/admin -env production migrate
Error running migrate up: Dirty database version 2021070800000000. Fix and force version.
You will need to manually fix the database. When you are sure that it is fixed, you can unset the dirty
column manually using an SQL client.
Installing The Developer Environment
To install the Traffic Ops Developer environment:
Clone the Traffic Control repository from GitHub. In most cases it is best to clone this directly into
GOPATH/src/github.com/apache/trafficcontrol
, as otherwise the Go implementation will not function properly.Install any required Go dependencies - the suggested method is using go-get(1).
# assuming current working directory is the repository root go mod vendor -v
Set up a role (user) in PostgreSQL
Use the
reset
andupgrade
command
s of admin (see app/db/admin for usage) to set up thetraffic_ops
database(s) and optionally with the--trafficvault
option to set up thetraffic_vault
database(s).Run the
traffic_ops/install/bin/postinstall
script, it will prompt for information like the default user login credentials.To run Traffic Ops, follow the instructions in Running.
app/db/traffic_vault_migrate
The traffic_vault_migrate
tool - located at traffic_ops/app/db/traffic_vault_migrate/traffic_vault_migrate.go
in the Apache Traffic Control repository -
is used to transfer TV keys between database servers. It interfaces directly with each backend so Traffic Ops/Vault being available is not a requirement.
The tool assumes that the schema for each backend is already setup as according to the admin setup.
Usage
traffic_vault_migrate [-cdhmr] [-e value] [-f value] [-g value] [-i value] [-l value] [-o value] [-t value]
- -c, --compare
Compare ‘to’ and ‘from’ backend keys. Will fetch keys from the dbs of both ‘to’ and ‘from’, sorts them by cdn/ds/version and does a deep comparison.
- -d, --dump
Write keys (from ‘from’ server) to disk in the folder ‘dump’ with the unix permissions 0640.
Warning
This can write potentially sensitive information to disk, use with care.
- -e LEVEL, --logLevel=LEVEL
Print everything at above specified log level (error|warning|info|debug|event) [info]
- -f CFG, --fromCfgPath=CFG
From server config file [“riak.json”]
- -g CFG, --toCfgPath=CFG
To server config file [“pg.json”]
- -h, --help
Displays usage information
- -i DIR, --fill DIR
Insert data into to server with data in this directory
- -l CFG, --logCfg CFG
Log configuration file
Note
Mutually exclusive with
-e
/--logLevel
- -o TYPE, --toType=TYPE
From server types (Riak|PG) [PG]
- -m, --noConfirm
Don’t require confirmation before inserting records
- -r, --dry
Do not perform writes. Will do a basic output of the keys on the ‘from’ backend.
- -t TYPE, --fromType=TYPE
From server types (Riak|PG) [Riak]
Riak
riak.json
- user:
The username used to log into the Riak server.
- password:
The password used to log into the Riak server.
- host:
The hostname for the Riak server.
- port:
The port for which the Riak server is listening for protobuf connections.
- timeout:
The number of seconds to wait for each operation.
- insecure:
(Optional) Determines whether to verify insecure certificates.
- tlsVersion:
(Optional) Max TLS version supported. Valid values are “10”, “11”, “12”, “13”.
Postgres
traffic_vault_migrate will properly handle both encryption and decryption of postgres data as that is done on the client side.
pg.json
- user:
The username used to log into the PG server.
- password:
The password for the user to log into the PG server.
- database:
The database to connect to.
- port:
The port on which the PG server is listening.
- host:
The hostname of the PG server.
- sslmode:
The ssl settings for the client connection, explanation here. Options are ‘disable’, ‘allow’, ‘prefer’, ‘require’, ‘verify-ca’ and ‘verify-full’
- aesKey:
The base64 encoding of a 16, 24, or 32 bit AES key.
Logging
The log configuration file has the structure:
- error_log:
Where to output error messages (stderr|stdout|null)
- warning_log:
Where to output warning messages (stderr|stdout|null)
- info_log:
Where to output info messages (stderr|stdout|null)
- debug_log:
Where to output error messages (stderr|stdout|null)
- event_log:
Where to output error messages (stderr|stdout|null)
Adding a Migration Backend
To add a plugin, implement the traffic_vault_migrate.go:TVBackend interface and add the backend to the returned values in github.com/apache/trafficcontrol/v8/traffic_ops/app/db/traffic_vault_migrate.supportBackends
.
Test Cases
Go Tests
Many (but not all) endpoint handlers and utility packages in the Go code-base define Go unit tests that can be run with go-test(1). There are integration tests for the Traffic Ops Go client in traffic_ops/testing/api/
.
cd traffic_ops/traffic_ops_golang
# run just one test
go test ./about
# run all of the tests
go test ./...
There are a few prerequisites to running the Go client integration tests:
A PostgreSQL server must be accessible and have a Traffic Ops database schema set up (though not necessarily populated with anything).
A running Traffic Ops Go implementation instance must be accessible - it shouldn’t be necessary to also be running the Perl implementation.
Note
For testing purposes, SSL certificates are not verified, so self-signed certificates will work fine.
Note
It is highly recommended that the Traffic Ops instance be run on the same machine as the integration tests, as otherwise network latency can cause the tests to exceed their threshold time limit of ten minutes.
The integration tests are run using go-test(1), with two configuration options available.
Note
It should be noted that the integration tests will output thousands of lines of highly repetitive text not directly related to the tests its running at the time - even if the -v
flag is not passed to go-test(1). This problem is tracked by Issue #4017.
Warning
Running the tests will wipe the connected database clean, so do not ever run it on an instance of Traffic Ops that holds meaningful data.
- --cfg CONFIG
Specify the path to the Test Configuration File. If not specified, it will attempt to read a file named
traffic-ops-test.conf
in the working directory.See also
Configuring the Integration Tests for a detailed explanation of the format of this configuration file.
- --fixtures FIXTURES
Specify the path to a file containing static data for the tests to use. This should almost never be used, because many of the tests depend on the data having a certain content and structure. If not specified, it will attempt to read a file named
tc-fixtures.json
in the working directory.
- --includeSystemTests {no|yes}
Specify whether to run tests that depend on additional components like an SMTP server or a Traffic Vault server. Default:
no
Configuring the Integration Tests
Configuration is mainly done through the configuration file passed as --cfg
, but is also available through the following environment variables.
In addition to the variables described here, the integration tests support identifying the network location of the Traffic Ops instance via TO_URL
.
- SESSION_TIMEOUT_IN_SECS
Sets the timeout of requests made to the Traffic Ops instance, in seconds.
- TODB_DESCRIPTION
An utterly cosmetic variable which, if set, gives a description of the PostgreSQL database to which the tests will connect. This has no effect except possibly changing one line of debug output.
- TODB_HOSTNAME
If set, will define the FQDN at which the PostgreSQL server to be used by the tests resides[1].
- TODB_PASSWORD
If set, defines the password to use when authenticating with the PostgreSQL server.
- TODB_SSL
If set, must be one of the following values:
- true
The PostgreSQL server to which the tests will connect uses SSL on the port on which it will be contacted.
- false
The PostgreSQL server to which the tests will connect does not use SSL on the port on which it will be contacted.
- TODB_TYPE
If set, tells the database driver used by the tests the kind of SQL database to which they are connecting[1]. This author has no idea what will happen if this is set to something other than
Pg
, but it’s possible the tests will fail to run. Certainly never do it.
- TODB_USER
If set, defines the user as whom to authenticate with the PostgreSQL server.
- TO_USER_ADMIN
If set, will define the name of a user with the “admin” Role that will be created by the tests[2].
- TO_USER_DISALLOWED
If set, will define the name of a user with the “disallowed” Role that will be created by the tests[2].
- TO_USER_EXTENSION
If set, will define the name of a user with the “extension” Role that will be created by the tests[2].
Caution
Due to legacy constraints, the only truly safe value for this is
extension
- anything else could cause the tests to fail.
- TO_USER_FEDERATION
If set, will define the name of a user with the “federation” Role that will be created by the tests[2].
- TO_USER_OPERATIONS
If set, will define the name of a user with the “operations” Role that will be created by the tests[2].
- TO_USER_PASSWORD
If set, will define the password used by all users created by the tests. This does not need to be the password of any pre-existing user.
- TO_USER_PORTAL
If set, will define the name of a user with the “portal” Role that will be created by the tests[2].
- TO_USER_READ_ONLY
If set, will define the name of a user with the “read-only” Role that will be created by the tests[2].
Test Configuration File
The configuration file for the tests (defined by --cfg
) is a JSON-encoded object with the following properties.
Warning
Many of these configuration options are overridden by variables in the execution environment. Where this is a problem, there is an associated warning. In general, this issue is tracked by Issue #3975.
- default:
An object containing sub-objects relating to default configuration settings for connecting to external resources during testing
- logLocations:
An object containing key/value pairs where the keys are log levels and each associated value is the file location to which logs of that level will be written. The allowed values respect the reserved special names used by the
github.com/apache/trafficcontrol/v8/lib/go-log
package. Omitted keys are treated as though their values werenull
, in which case that level is written to /dev/null. The allowed keys are:debug
error
event
info
warning
- session:
An object containing key/value pairs that define the default settings used by Traffic Ops “session” connections
- timeoutInSecs:
At the time of this writing this is the only meaningful configuration option that may be present under
session
. It specifies the timeouts used by client connections during testing as an integer number of seconds. The default if not specified (or overridden) is 0, meaning no limit.Warning
This configuration is overridden by
SESSION_TIMEOUT_IN_SECS
.
- trafficOps:
An object containing information that defines the running Traffic Ops instance to use in testing.
- password:
This password will be used for all created users used by the test suite - it does not need to be the password of any pre-existing user. The default if not specified (or overridden) is an empty string, which may or may not cause problems.
Warning
This is overridden by
TO_USER_PASSWORD
.- URL:
The network location of the running Traffic Ops server, including schema, hostname and optionally port number e.g.
https://localhost:6443
.Warning
This is overridden by
TO_URL
.- users:
An object containing key-value pairs where the keys are the names of Roles and the values are the usernames of users that will be created with the associated Role for testing purposes. There are very few good reasons why the values should not just be the same as the keys. The default for any missing (and not overridden) key is the empty string which is won’t work so please don’t leave any undefined. The allowed keys are:
admin
Warning
The value of this key is overridden by
TO_USER_ADMIN
.disallowed
Warning
The value of this key is overridden by
TO_USER_DISALLOWED
.extension
Warning
The value of this key is overridden by
TO_USER_EXTENSION
.federation
Warning
The value of this key is overridden by
TO_USER_FEDERATION
.operations
Warning
The value of this key is overridden by
TO_USER_OPERATIONS
.portal
Warning
The value of this key is overridden by
TO_USER_PORTAL
.readOnly
Warning
The value of this key is overridden by
TO_USER_READ_ONLY
.
- trafficOpsDB:
An object containing information that defines the database to use in testing[1].
- dbname:
The name of the database to which the tests will connect[1].
Warning
This is overridden by
TODB_NAME
.- description:
An utterly cosmetic option that need not exist at all which, if set, gives a description of the database to which the tests will connect. This has no effect except possibly changing one line of debug output.
Warning
This is overridden by
TODB_DESCRIPTION
- hostname:
The FQDN of the server on which the database is running[1]
Warning
This is overridden by
TODB_HOSTNAME
.- password:
The password to use when authenticating with the database
Warning
This is overridden by
TODB_PASSWORD
.- port:
The port on which the database listens for connections[1] - as a string
Warning
This is overridden by
TODB_PORT
.- type:
The “type” of database being used[1]. This should never be set to anything besides
"Pg"
, anything else results in undefined behavior (although it’s equally possible that it simply won’t have any effect).Warning
This is overridden by
TODB_TYPE
.- ssl:
An optional boolean value that defines whether or not the database uses SSL encryption for its connections - default if not specified (or overridden) is
false
Warning
This is overridden by
TODB_SSL
.- user:
The name of the user as whom to authenticate with the database
Warning
This is overridden by
TODB_USER
.
Writing New Endpoints
Note
Most new endpoints are accompanied by database schema changes which necessitate a new migration under traffic_ops/app/db/migrations
and database best-practices are not discussed in this section.
See also
This section contains a quick overview of API endpoint development; for the full guidelines for API endpoints, consult API Guidelines.
The first thing to consider when writing a new endpoint is what the requests it will serve will look like. It’s recommended that new endpoints avoid using “path parameters” when possible, and instead try to utilize request bodies and/or query string parameters. For example, instead of /foos/{{ID}}
consider simply /foos
with a supported id
query parameter. The request methods should be restricted to the following, and respect each method’s associated meaning.
- DELETE
Removes a resource or one or more of its representations from the server. This should always be the method used when deleting objects.
- GET
Retrieves a representation of some resource. This should always be used for read-only operations and note that the requesting client never expects the state of the server to change as a result of a request using the GET method.
- POST
Requests that the server process some passed data. This is used most commonly to create new objects on the server, but can also be used more generally e.g. with a request for regenerating encryption keys. Although this isn’t strictly creating new API resources, it does change the state of the server and so this is more appropriate than GET.
- PUT
Places a new representation of some resource on the server. This is typically used for updating existing objects. For creating new representations/objects, use POST instead. When using PUT note that clients expect it to be idempotent, meaning that subsequent identical PUT requests should result in the same server state. What this means is that it’s standard to require that all of the information defining a resource be provided for each request even if the vast majority of it isn’t changing.
The HEAD and OPTIONS request methods have default implementations for any properly defined Traffic Ops API route, and so should almost never be defined explicitly. Other request methods (e.g. CONNECT) are currently unused and ought to stay that way for the time being.
Note
Utilizing the PATCH method is unfeasible at the time of this writing but progress toward supporting it is being made, albeit slowly in the face of other priorities.
See also
The MDN’s documentation on the various HTTP request methods.
The final step of creating any Traffic Ops API endpoint is to write documentation for it. When doing so, be sure to follow all of the guidelines laid out in Documentation Guidelines. If documentation doesn’t exist for new functionality then it has accomplished nothing because no one using Traffic Control will know it exists. Omitted documentation is how a project winds up with a dozen different API endpoints that all do essentially the same thing.
Framework Options
The Traffic Ops code base offers two basic frameworks for defining a new endpoint. Either one may be used at the author’s discretion (or even neither if desired and appropriate - though that seems unlikely).
Generic “CRUDer”
The “Generic ‘CRUDer’”, as it’s known, is a pattern of API endpoint development that principally involves defining a type
that implements the github.com/apache/trafficcontrol/v8/traffic_ops/traffic_ops_golang/api.CRUDer
interface. A description of what that entails is best left to the actual GoDoc documentation.
See also
The github.com/apache/trafficcontrol/v8/traffic_ops/traffic_ops_golang/api.GenericCreate
, github.com/apache/trafficcontrol/v8/traffic_ops/traffic_ops_golang/api.GenericDelete
, github.com/apache/trafficcontrol/v8/traffic_ops/traffic_ops_golang/api.GenericRead
, and github.com/apache/trafficcontrol/v8/traffic_ops/traffic_ops_golang/api.GenericUpdate
helpers are often used to provide the default operations of creating, deleting, reading, and updating objects, respectively. When the API endpoint being written is only meant to perform these basic operations on an object or objects stored in the database, these should be totally sufficient.
This method offers a lot of functionality “out-of-the-box” as compared to the APIInfo method, but because of that is also restrictive. For example, it is not possible to write an endpoint that returns data not encoded as JSON using this method. That’s an uncommon use-case, but not unheard-of.
This method is best used for basic creation, reading, update, and deletion operations performed on simple objects with no structural differences across API versions.
APIInfo
Endpoint handlers can also be defined by simply implementing the net/http.HandlerFunc
interface. The net/http.Request
reference passed into such handlers provides identifying information for the authenticated user (where applicable) in its context.
To easily obtain the information needed to identify a user and their associated permissions, as well as server configuration information and a database transaction handle, authors should use the github.com/apache/trafficcontrol/v8/traffic_ops/traffic_ops_golang/api.NewInfo
function which will return all of that information in a single structure as well as any errors encountered during the process and an appropriate HTTP response code in case of such errors.
This method offers fine control over the endpoint’s logic, but tends to be much more verbose than the endpoints written using the Generic “CRUDer” method. For example, a handler for retrieving an object from the database and returning it to the requesting client encoded as JSON can be twenty or more lines of code, whereas a single call to github.com/apache/trafficcontrol/v8/traffic_ops/traffic_ops_golang/api.GenericCreate
provides equivalent functionality.
This method is best used when requests are meant to have extensive side-effects, are performed on unusually structured objects, need fine control of the HTTP headers/options, or operate on objects that have different structures or meanings across API versions.
Extensions
What’s typically meant by “extension” in the context of Traffic Ops is a Check Extensions which provides data for server “checks” which can be viewed in Traffic Portal under . This type of extension need not know nor even care which implementation it is being used with, as it interacts with Traffic Ops through the Traffic Ops API.
Traffic Ops supports overrides or new definitions for non-standard Traffic Ops API routes. This type of “extension” is typically reffered to as a “plugin,” and they are described in Go Plugins.
Go Plugins
A plugin is defined by a Go source file in the traffic_ops/traffic_ops_golang/plugin
directory, which is expected to be named plugin name.go
. A plugin is registered to Traffic Ops by a call to github.com/apache/trafficcontrol/v8/traffic_ops/traffic_ops_golang/plugin.AddPlugin
in the source file’s special init
function.
A plugin is only enabled at runtime if its name is present in the cdn.conf file’s traffic_ops_golang.plugins
array.
Each plugin may also define any, all, or none of the lifecycle hooks provided: load
, startup
, and onRequest
- load
The
load
function of a plugin, if defined, needs to implement thegithub.com/apache/trafficcontrol/v8/traffic_ops/traffic_ops_golang/plugin.LoadFunc
interface, and will be run when the server starts and after configuration has been loaded. It will be passed the plugins own configuration as it was defined in the cdn.conf file’straffic_ops_golang.plugin_config
map.- onRequest
The
onRequest
function of a plugin, if defined, needs to implement thegithub.com/apache/trafficcontrol/v8/traffic_ops/traffic_ops_golang/plugin.OnRequestFunc
interface, and will be called on every request made to the Traffic Ops API. Because of this, it’s imperative that the function exit as soon as possible. Note that once one plugin reports that it has served the request, no others will be tried. The order in which plugins are tried is defined by their order in thetraffic_ops_golang.plugins
array of the cdn.conf configuration file.See also
It’s very common for this function to behave like a Traffic Ops API endpoint, so when writing a plugin it may be useful to review Writing New Endpoints.
- startup
Like
load
, thestartup
function of a plugin, if defined, will be called when the server starts and after configuration has been loaded. Unlikeload
, however, this function should implement thegithub.com/apache/trafficcontrol/v8/traffic_ops/traffic_ops_golang/plugin.StartupFunc
interface and will be passed in the entirety of the server’s configuration, including its own configuration and any shared plugin configuration data as defined in the cdn.conf file’straffic_ops_golang.plugin_shared_config
map.
Example
An example “Hello World” plugin that serves the /_hello
request path by just writing “Hello World” in the body of a 200 OK response back to the client is provided in traffic_ops/traffic_ops_golang/plugin/hello_world.go
:
1package plugin
2
3/*
4 Licensed under the Apache License, Version 2.0 (the "License");
5 you may not use this file except in compliance with the License.
6 You may obtain a copy of the License at
7
8 http://www.apache.org/licenses/LICENSE-2.0
9
10 Unless required by applicable law or agreed to in writing, software
11 distributed under the License is distributed on an "AS IS" BASIS,
12 WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13 See the License for the specific language governing permissions and
14 limitations under the License.
15*/
16
17import (
18 "strings"
19)
20
21func init() {
22 AddPlugin(10000, Funcs{onRequest: hello}, "example HTTP plugin", "1.0.0")
23}
24
25const HelloPath = "/_hello"
26
27func hello(d OnRequestData) IsRequestHandled {
28 if !strings.HasPrefix(d.R.URL.Path, HelloPath) {
29 return RequestUnhandled
30 }
31 d.W.Header().Set("Content-Type", "text/plain")
32 d.W.Write([]byte("Hello, World!"))
33 return RequestHandled
34}
Check Extensions
Check Extensions allow you to add custom checks to the view.
Extensions are managed using the $TO_HOME/bin/extensions
command line script
See also
For more information see Managing Traffic Ops Extensions.