Traffic Ops

At one point, Traffic Ops was a collection of Perl scripts, and while the current program is written in Go, many of its tools and utilities are still written in Perl.

Introduction

Traffic Ops at its core is mainly a PostgreSQL database used to store configuration information for ATC, and a set of RESTful API endpoints for interacting with and manipulating that information. It also serves as the single point of authentication for ATC components (that is, when one hears “user” in an ATC context it nearly always means a “user” as configured in Traffic Ops) and provides interfaces to other ATC components by proxy. Additionally, there is some miscellaneous, at times obscure functionality to Traffic Ops, such as generating arbitrary Linux system images.

Software Requirements

Traffic Ops is only supported on CentOS 7+ systems (although many developers do use Mac OS with some success). Here are the requirements:

PostgreSQL 13.2 - the machine where Traffic Ops is running must have the client tool set (e.g. psql(1)), but the actual database can be run anywhere so long as it is accessible.
Note

Prior to version 13.2, Traffic Ops used version 9.6. For upgrading an existing Mac OS Homebrew-based PostgreSQL instance, you can use Homebrew to easily upgrade from 9.6 to 13.2:
brew services stop postgresql brew upgrade postgresql brew postgresql-upgrade-database brew cleanup postgresql@9.6 brew services start postgresql
openssl(1SSL) is recommended to generate server certificates, though not strictly required if certificates can be obtained by other means.
Some kind of SMTP server is required for certain Traffic Ops API endpoints to work, but for purposes unrelated to them an SMTP server is not required. CDN in a Box comes with a relayless SMTP server for testing (you can view the emails that Traffic Ops sends, but they aren’t sent anywhere outside CDN-in-a-Box).

Tip

Alternatively, development and testing can be done using CDN in a Box - albeit somewhat more slowly.

Perl Script Requirements

Not much code is still in Perl, but for the scripts the following are needed:

Perl 5.10.1
libpcap and libpcap development library - usually libpcap-dev or libpcap-devel in your system’s native package manager.
libpq and libpq development library - usually libpq-dev or libpq-devel in your system’s native package manager.
The JSON Perl pod from CPAN
The JSON::PP Perl pod from CPAN

Developers should use Perltidy to format their Perl code.

#137 Example Perltidy Configuration (usually in HOME/.perltidyrc)

-l=156
-et=4
-t
-ci=4
-st
-se
-vt=0
-cti=0
-pt=1
-bt=1
-sbt=1
-bbt=1
-nsfs
-nolq
-otr
-aws
-wls="= + - / * ."
-wrs=\"= + - / * .\"
-wbb="% + - * / x != == >= <= =~ < > | & **= += *= &= <<= &&= -= /= |= + >>= ||= .= %= ^= x="

Go Program Requirements

Go 1.19 or later
If the system’s Go compiler doesn’t provide it implicitly, also note that all Go code in the ATC repository should be formatted using gofmt

All Go code dependencies are managed through the go.mod, go.sum, and vendor/modules.txt files. With the exception of golang.org/x packages (see below), module dependencies in vendor/ are tracked in Git and should thus be available without any extra work - and any new dependencies should be properly “vendored” into that same, top-level directory. No other vendor/ directories exist, as Go modules only supports a single vendor directory.

Per the Go language standard’s authoritative source’s recommendation, all sub-packages of golang.org/x are treated as a part of the compiler, and so need not ever be “vendored” as though they were an external dependency. These dependencies are not listed explicitly here, so it is strongly advised that they be fetched using go-get(1) rather than downloaded by hand.

Tip

All new dependencies need to be subject to community review to ensure necessity (because it will be added in its entirety to the repository, after all) and license compliance via the developer mailing list.

Traffic Ops Project Tree Overview

traffic_ops/ - The root of the Traffic Ops project
- app/ - Holds most of the Perl code base, though many of the files contained herein are also used by the Go implementation
  Note
  
  This directory is home to many things that no longer work as intended or have been superseded by other things.
  
  bin/ - Directory for scripts and tools, cron(8) jobs, etc.
  
  checks/ - Contains the Check Extensions scripts that are provided by default
  
  db/ - Contains scripts that manipulate the database beyond the scope of setup, migration, and seeding
  
  tests/ - Integration and unit test scripts for automation purposes - in general this has been superseded by traffic_ops/testing/api/
  
  conf/ - Aggregated configuration for Traffic Ops. For convenience, different environments for the app/db/admin tool are already set up
  
  development/ - Configuration files for the “development” environment
  
  integration/ - Configuration files for the “integration” environment
  
  misc/ - Miscellaneous configuration files.
  
  production/ - Configuration files for the “production” environment
  
  test/ - Configuration files for the “test” environment
  
  db/ - Database setup, seeding, and upgrade/downgrade helpers
  
  migrations/ - Database migration files
  
  tools/ - Contains helper scripts for easing upgrade transitions when selective data manipulation must be done to achieve a desirable state
  
  script/ - Mojolicious bootstrap/startup scripts.
  
  templates/ - Mojolicious Embedded Perl (template name.ep) files for the now-removed Traffic Ops UI
- build/ - Contains files that are responsible for packaging Traffic Ops into an RPM file - and also for doing the same with ORT
- etc/ - Configuration files for various systems associated with running production instances of Traffic Ops, which are installed under /etc by the Traffic Ops RPM
  cron.d/ - Holds specifications for cron(8) jobs that need to be run periodically on Traffic Ops servers
  
  Note
  
  At least one of these jobs expects itself to be run on a server that has the Perl implementation of Traffic Ops installed under /opt/traffic_ops/. Nothing terrible will happen if that’s not true, just that it/they won’t work. Installation using the RPM will set up all of these kinds of things up automatically.
  
  init.d/ - Contains the old, initscripts-based job control for Traffic Ops
  
  logrotate.d/ - Specifications for the Linux logrotate(8) utility for Traffic Ops log files
  
  profile.d/traffic_ops.sh - Sets up common environment variables for working with Traffic Ops
- install/ - Contains all of the resources necessary for a full install of Traffic Ops
  bin/ - Binaries related to installing Traffic Ops, as well as installing its prerequisites, certificates, and database
  
  data/ - Contains things that need to be accessible by the running server for certain functionality - typically installed to /var/www/data by the RPM (hence the name).
  
  etc/ - This directory left empty; it’s used to contain post-installation extensions and resources
  
  lib/ - Contains libraries used by the various installation binaries
- ort/ - Contains ORT and ATS configuration file-generation logic and tooling
- testing/ - Holds utilities for testing the Traffic Ops API
  api/ - Integration testing for the Traffic Ops Go client and Traffic Ops
- traffic_ops_golang/ - The root of the Go implementation’s code-base
  Note
  
  The vast majority of subdirectories of traffic_ops/traffic_ops_golang/ contain handlers for the Traffic Ops API, and are named according to the endpoint they handle. What follows is a list of subdirectories of interest that have a special role (i.e. don’t handle a Traffic Ops API endpoint).
  
  See also
  
  The GoDoc documentation for this package
  
  api/ - A library for use by Traffic Ops API handlers that provides helpful utilities for common tasks like obtaining a database transaction handle or accessing Traffic Ops configuration
  
  auth/ - Contains definitions of privilege levels and access control code used in routing and provides a library for dealing with password and token-based authentication
  
  config/ - Defines configuration structures and methods for reading them in from files
  
  dbhelpers/ - Assorted utilities that provide functionality for common database tasks, e.g. “Get a user by email”
  
  plugin/ - The Traffic Ops plugin system, with examples
  
  trafficvault/ - This package provides the Traffic Vault interface and associated backend implementations for other handlers to interact with Traffic Vault.
  
  routing/ - Contains logic for mapping all of the Traffic Ops API endpoints to their handlers, as well as proxying requests back to the Perl implementation and managing plugins, and also provides some wrappers around registered handlers that set common HTTP headers and connection options
  
  swaggerdocs/ A currently abandoned attempt at defining the Traffic Ops API using Swagger - it may be picked up again at some point in the (distant) future
  
  tenant/ - Contains utilities for dealing with Tenantable resources, particularly for checking for permissions
  
  tocookie/ - Defines the method of generating the mojolicious cookie used by Traffic Ops for authentication
  
  vendor/ - contains “vendored” Go packages from third party sources
- v3-client - The official Traffic Ops Go client package for working with the version 3 Traffic Ops API.
- v4-client - The official Traffic Ops Go client package for working with the version 4 Traffic Ops API.
- v5-client - The official Traffic Ops Go client package for working with the version 5 Traffic Ops API.
- vendor/ - contains “vendored” Go packages from third party sources

app/db/admin

The app/db/admin binary is for use in managing the Traffic Ops (and Traffic Vault PostgreSQL backend) database tables. This essentially serves as a front-end for Migrate, though dbconf.yml comes from Goose, which Traffic Ops used to use before switching to Migrate.

Note

For proper resolution of configuration and SOL statement files, it’s recommended that this binary be run from the app directory

Usage

db/admin [options] command

Options and Arguments

--env ENVIRONMENT

An optional environment specification that causes the database configuration to be read out of the corresponding section of the app/db/dbconf.yml configuration file. One of:

development
integration
production
test

admin sets MOJO_MODE to the value of the environment as specified by this option. (Default: development)

--trafficvault: When used, commands will be run against the Traffic Vault PostgreSQL backend database as specified in the app/db/trafficvault/dbconf.yml configuration file.

command

The command specifies the operation to be performed on the database. It must be one of:

createdb: Creates the database for the current environment
create_migration: Creates a pair of timestamped up/down migrations titled NAME.
create_user: Creates the user defined for the current environment
dbversion: Displays the database version that results from the current sequence of migrations
down: Rolls back a single migration from the current version
drop: Drops the database for the current environment
drop_user: Drops the user defined for the current environment
load_schema: Sets up the database for the current environment according to the SQL statements in app/db/create_tables.sql or app/db/trafficvault/create_tables.sql if the --trafficvault option is used
migrate: Runs a migration on the database for the current environment
patch: Patches the database for the current environment using the SQL statements from the app/db/patches.sql. This command is not supported when using the --trafficvault option
redo: Rolls back the most recently applied migration, then run it again
reset: Creates the user defined for the current environment, drops the database for the current environment, creates a new one, loads the schema into it, and runs a single migration on it
seed: Executes the SQL statements from the app/db/seeds.sql file for loading static data. This command is not supported when using the --trafficvault option
show_users: Displays a list of all users registered with the PostgreSQL server
status: Deprecated, status is now an alias for dbversion and will be removed in a future Traffic Control release.
upgrade: Performs a migration on the database for the current environment, then seeds it and patches it using the SQL statements from the app/db/patches.sql file

#138 Example Usage

db/admin --env=test reset

The environments are defined in the traffic_ops/app/db/dbconf.yml file, and the name of the database generated will be the name of the environment for which it was created. If the --trafficvault option is used, the app/db/trafficvault/dbconf.yml file defines this information.

Resolving Migration Failures

If you encounter an error running a migration, you will see a message like

#139 db/admin error example

[root@trafficops app]# db/admin -env production migrate
Error running migrate up: migration failed: syntax error at or near "This_is_a_syntax_error" (column 1) in line 18: /*

That means that the migration timestamp in the version column of the schema_migrations table has been updated to the version of the migration that failed, but the dirty column is also set, and if you try to run another migration (either up or down), you will see

#140 db/admin error migrating when the database version is dirty

[root@trafficops app]# db/admin -env production migrate
Error running migrate up: Dirty database version 2021070800000000. Fix and force version.

You will need to manually fix the database. When you are sure that it is fixed, you can unset the dirty column manually using an SQL client.

Installing The Developer Environment

To install the Traffic Ops Developer environment:

Clone the Traffic Control repository from GitHub. In most cases it is best to clone this directly into GOPATH/src/github.com/apache/trafficcontrol, as otherwise the Go implementation will not function properly.
Install any required Go dependencies - the suggested method is using go-get(1).
#141 Install Go Development Dependencies
# assuming current working directory is the repository root go mod vendor -v
Set up a role (user) in PostgreSQL

See also

PostgreSQL instructions on setting up a database.
Use the reset and upgrade commands of admin (see app/db/admin for usage) to set up the traffic_ops database(s) and optionally with the --trafficvault option to set up the traffic_vault database(s).
Run the traffic_ops/install/bin/postinstall script, it will prompt for information like the default user login credentials.
To run Traffic Ops, follow the instructions in Running.

app/db/traffic_vault_migrate

The traffic_vault_migrate tool - located at traffic_ops/app/db/traffic_vault_migrate/traffic_vault_migrate.go in the Apache Traffic Control repository - is used to transfer TV keys between database servers. It interfaces directly with each backend so Traffic Ops/Vault being available is not a requirement. The tool assumes that the schema for each backend is already setup as according to the admin setup.

Usage

traffic_vault_migrate [-cdhmr] [-e value] [-f value] [-g value] [-i value] [-l value] [-o value] [-t value]

-c, --compare: Compare ‘to’ and ‘from’ backend keys. Will fetch keys from the dbs of both ‘to’ and ‘from’, sorts them by cdn/ds/version and does a deep comparison.

Note

Mutually exclusive with -r/--dry

-d, --dump: Write keys (from ‘from’ server) to disk in the folder ‘dump’ with the unix permissions 0640.

Warning

This can write potentially sensitive information to disk, use with care.

Note

Mutually exclusive with -l/--fill

-e LEVEL, --logLevel=LEVEL: Print everything at above specified log level (error|warning|info|debug|event) [info]

Note

Mutually exclusive with -l/--logCfg

-f CFG, --fromCfgPath=CFG: From server config file [“riak.json”]

-g CFG, --toCfgPath=CFG: To server config file [“pg.json”]

-h, --help: Displays usage information

-i DIR, --fill DIR: Insert data into to server with data in this directory

Note

Mutually exclusive with -d/--dump

-l CFG, --logCfg CFG: Log configuration file

Note

Mutually exclusive with -e/--logLevel

-o TYPE, --toType=TYPE: From server types (Riak|PG) [PG]

-m, --noConfirm: Don’t require confirmation before inserting records

-r, --dry: Do not perform writes. Will do a basic output of the keys on the ‘from’ backend.

Note

Mutually exclusive with -c/--compare

-t TYPE, --fromType=TYPE: From server types (Riak|PG) [Riak]

Riak

riak.json

user

The username used to log into the Riak server.

password

The password used to log into the Riak server.

host

The hostname for the Riak server.

port

The port for which the Riak server is listening for protobuf connections.

timeout

The number of seconds to wait for each operation.

insecure

(Optional) Determines whether to verify insecure certificates.

tlsVersion

(Optional) Max TLS version supported. Valid values are “10”, “11”, “12”, “13”.

Postgres

traffic_vault_migrate will properly handle both encryption and decryption of postgres data as that is done on the client side.

pg.json

user

The username used to log into the PG server.

password

The password for the user to log into the PG server.

database

The database to connect to.

port

The port on which the PG server is listening.

host

The hostname of the PG server.

sslmode

The ssl settings for the client connection, explanation here. Options are ‘disable’, ‘allow’, ‘prefer’, ‘require’, ‘verify-ca’ and ‘verify-full’

aesKey

The base64 encoding of a 16, 24, or 32 bit AES key.

Logging

The log configuration file has the structure:

error_log

Where to output error messages (stderr|stdout|null)

warning_log

Where to output warning messages (stderr|stdout|null)

info_log

Where to output info messages (stderr|stdout|null)

debug_log

Where to output error messages (stderr|stdout|null)

event_log

Where to output error messages (stderr|stdout|null)

Adding a Migration Backend

To add a plugin, implement the traffic_vault_migrate.go:TVBackend interface and add the backend to the returned values in github.com/apache/trafficcontrol/traffic_ops/app/db/traffic_vault_migrate.supportBackends.

Test Cases

Go Tests

Many (but not all) endpoint handlers and utility packages in the Go code-base define Go unit tests that can be run with go-test(1). There are integration tests for the Traffic Ops Go client in traffic_ops/testing/api/.

#142 Sample Run of Go Unit Tests

cd traffic_ops/traffic_ops_golang

# run just one test
go test ./about

# run all of the tests
go test ./...

There are a few prerequisites to running the Go client integration tests:

A PostgreSQL server must be accessible and have a Traffic Ops database schema set up (though not necessarily populated with anything).
A running Traffic Ops Go implementation instance must be accessible - it shouldn’t be necessary to also be running the Perl implementation.

Note

For testing purposes, SSL certificates are not verified, so self-signed certificates will work fine.

Note

It is highly recommended that the Traffic Ops instance be run on the same machine as the integration tests, as otherwise network latency can cause the tests to exceed their threshold time limit of ten minutes.

The integration tests are run using go-test(1), with two configuration options available.

Note

It should be noted that the integration tests will output thousands of lines of highly repetitive text not directly related to the tests its running at the time - even if the -v flag is not passed to go-test(1). This problem is tracked by Issue #4017.

Warning

Running the tests will wipe the connected database clean, so do not ever run it on an instance of Traffic Ops that holds meaningful data.

--cfg CONFIG: Specify the path to the Test Configuration File. If not specified, it will attempt to read a file named traffic-ops-test.conf in the working directory.

See also

Configuring the Integration Tests for a detailed explanation of the format of this configuration file.

--fixtures FIXTURES: Specify the path to a file containing static data for the tests to use. This should almost never be used, because many of the tests depend on the data having a certain content and structure. If not specified, it will attempt to read a file named tc-fixtures.json in the working directory.

--includeSystemTests {no|yes}: Specify whether to run tests that depend on additional components like an SMTP server or a Traffic Vault server. Default: no

Configuring the Integration Tests

Configuration is mainly done through the configuration file passed as --cfg, but is also available through the following environment variables.

In addition to the variables described here, the integration tests support identifying the network location of the Traffic Ops instance via TO_URL.

SESSION_TIMEOUT_IN_SECS: Sets the timeout of requests made to the Traffic Ops instance, in seconds.

TODB_DESCRIPTION: An utterly cosmetic variable which, if set, gives a description of the PostgreSQL database to which the tests will connect. This has no effect except possibly changing one line of debug output.

TODB_HOSTNAME: If set, will define the FQDN at which the PostgreSQL server to be used by the tests resides1.

TODB_NAME: If set, will define the name of the database to which the tests will connect1.

TODB_PASSWORD: If set, defines the password to use when authenticating with the PostgreSQL server.

TODB_PORT: If set, defines the port on which the PostgreSQL server listens1.

TODB_SSL

If set, must be one of the following values:

true: The PostgreSQL server to which the tests will connect uses SSL on the port on which it will be contacted.
false: The PostgreSQL server to which the tests will connect does not use SSL on the port on which it will be contacted.

TODB_TYPE: If set, tells the database driver used by the tests the kind of SQL database to which they are connecting1. This author has no idea what will happen if this is set to something other than Pg, but it’s possible the tests will fail to run. Certainly never do it.

TODB_USER: If set, defines the user as whom to authenticate with the PostgreSQL server.

TO_USER_ADMIN: If set, will define the name of a user with the “admin” Role that will be created by the tests2.

TO_USER_DISALLOWED: If set, will define the name of a user with the “disallowed” Role that will be created by the tests2.

TO_USER_EXTENSION: If set, will define the name of a user with the “extension” Role that will be created by the tests2.

Caution

Due to legacy constraints, the only truly safe value for this is extension - anything else could cause the tests to fail.

TO_USER_FEDERATION: If set, will define the name of a user with the “federation” Role that will be created by the tests2.

TO_USER_OPERATIONS: If set, will define the name of a user with the “operations” Role that will be created by the tests2.

TO_USER_PASSWORD: If set, will define the password used by all users created by the tests. This does not need to be the password of any pre-existing user.

TO_USER_PORTAL: If set, will define the name of a user with the “portal” Role that will be created by the tests2.

TO_USER_READ_ONLY: If set, will define the name of a user with the “read-only” Role that will be created by the tests2.

Test Configuration File

The configuration file for the tests (defined by --cfg) is a JSON-encoded object with the following properties.

Warning

Many of these configuration options are overridden by variables in the execution environment. Where this is a problem, there is an associated warning. In general, this issue is tracked by Issue #3975.

default

An object containing sub-objects relating to default configuration settings for connecting to external resources during testing

logLocations

An object containing key/value pairs where the keys are log levels and each associated value is the file location to which logs of that level will be written. The allowed values respect the reserved special names used by the github.com/apache/trafficcontrol/lib/go-log package. Omitted keys are treated as though their values were null, in which case that level is written to /dev/null. The allowed keys are:

debug
error
event
info
warning

session

An object containing key/value pairs that define the default settings used by Traffic Ops “session” connections

timeoutInSecs: At the time of this writing this is the only meaningful configuration option that may be present under session. It specifies the timeouts used by client connections during testing as an integer number of seconds. The default if not specified (or overridden) is 0, meaning no limit.

Warning

This configuration is overridden by SESSION_TIMEOUT_IN_SECS.

trafficOps

An object containing information that defines the running Traffic Ops instance to use in testing.

password

This password will be used for all created users used by the test suite - it does not need to be the password of any pre-existing user. The default if not specified (or overridden) is an empty string, which may or may not cause problems.

Warning

This is overridden by TO_USER_PASSWORD.

URL

The network location of the running Traffic Ops server, including schema, hostname and optionally port number e.g. https://localhost:6443.

Warning

This is overridden by TO_URL.

users

An object containing key-value pairs where the keys are the names of Roles and the values are the usernames of users that will be created with the associated Role for testing purposes. There are very few good reasons why the values should not just be the same as the keys. The default for any missing (and not overridden) key is the empty string which is won’t work so please don’t leave any undefined. The allowed keys are:

admin

Warning

The value of this key is overridden by TO_USER_ADMIN.
disallowed

Warning

The value of this key is overridden by TO_USER_DISALLOWED.
extension

Warning

The value of this key is overridden by TO_USER_EXTENSION.
federation

Warning

The value of this key is overridden by TO_USER_FEDERATION.
operations

Warning

The value of this key is overridden by TO_USER_OPERATIONS.
portal

Warning

The value of this key is overridden by TO_USER_PORTAL.
readOnly

Warning

The value of this key is overridden by TO_USER_READ_ONLY.

trafficOpsDB

An object containing information that defines the database to use in testing1.

dbname: The name of the database to which the tests will connect1.

Warning

This is overridden by TODB_NAME.
description: An utterly cosmetic option that need not exist at all which, if set, gives a description of the database to which the tests will connect. This has no effect except possibly changing one line of debug output.

Warning

This is overridden by TODB_DESCRIPTION
hostname: The FQDN of the server on which the database is running1

Warning

This is overridden by TODB_HOSTNAME.
password: The password to use when authenticating with the database

Warning

This is overridden by TODB_PASSWORD.
port: The port on which the database listens for connections1 - as a string

Warning

This is overridden by TODB_PORT.
type: The “type” of database being used1. This should never be set to anything besides "Pg", anything else results in undefined behavior (although it’s equally possible that it simply won’t have any effect).

Warning

This is overridden by TODB_TYPE.
ssl: An optional boolean value that defines whether or not the database uses SSL encryption for its connections - default if not specified (or overridden) is false

Warning

This is overridden by TODB_SSL.
user: The name of the user as whom to authenticate with the database

Warning

This is overridden by TODB_USER.

Writing New Endpoints

Note

Most new endpoints are accompanied by database schema changes which necessitate a new migration under traffic_ops/app/db/migrations and database best-practices are not discussed in this section.

See also

This section contains a quick overview of API endpoint development; for the full guidelines for API endpoints, consult API Guidelines.

The first thing to consider when writing a new endpoint is what the requests it will serve will look like. It’s recommended that new endpoints avoid using “path parameters” when possible, and instead try to utilize request bodies and/or query string parameters. For example, instead of /foos/{{ID}} consider simply /foos with a supported id query parameter. The request methods should be restricted to the following, and respect each method’s associated meaning.

DELETE: Removes a resource or one or more of its representations from the server. This should always be the method used when deleting objects.
GET: Retrieves a representation of some resource. This should always be used for read-only operations and note that the requesting client never expects the state of the server to change as a result of a request using the GET method.
POST: Requests that the server process some passed data. This is used most commonly to create new objects on the server, but can also be used more generally e.g. with a request for regenerating encryption keys. Although this isn’t strictly creating new API resources, it does change the state of the server and so this is more appropriate than GET.
PUT: Places a new representation of some resource on the server. This is typically used for updating existing objects. For creating new representations/objects, use POST instead. When using PUT note that clients expect it to be idempotent, meaning that subsequent identical PUT requests should result in the same server state. What this means is that it’s standard to require that all of the information defining a resource be provided for each request even if the vast majority of it isn’t changing.

The HEAD and OPTIONS request methods have default implementations for any properly defined Traffic Ops API route, and so should almost never be defined explicitly. Other request methods (e.g. CONNECT) are currently unused and ought to stay that way for the time being.

Note

Utilizing the PATCH method is unfeasible at the time of this writing but progress toward supporting it is being made, albeit slowly in the face of other priorities.

The final step of creating any Traffic Ops API endpoint is to write documentation for it. When doing so, be sure to follow all of the guidelines laid out in Documentation Guidelines. If documentation doesn’t exist for new functionality then it has accomplished nothing because no one using Traffic Control will know it exists. Omitted documentation is how a project winds up with a dozen different API endpoints that all do essentially the same thing.

Framework Options

The Traffic Ops code base offers two basic frameworks for defining a new endpoint. Either one may be used at the author’s discretion (or even neither if desired and appropriate - though that seems unlikely).

Generic “CRUDer”

The “Generic ‘CRUDer’”, as it’s known, is a pattern of API endpoint development that principally involves defining a type that implements the github.com/apache/trafficcontrol/traffic_ops/traffic_ops_golang/api.CRUDer interface. A description of what that entails is best left to the actual GoDoc documentation.

See also

The github.com/apache/trafficcontrol/traffic_ops/traffic_ops_golang/api.GenericCreate, github.com/apache/trafficcontrol/traffic_ops/traffic_ops_golang/api.GenericDelete, github.com/apache/trafficcontrol/traffic_ops/traffic_ops_golang/api.GenericRead, and github.com/apache/trafficcontrol/traffic_ops/traffic_ops_golang/api.GenericUpdate helpers are often used to provide the default operations of creating, deleting, reading, and updating objects, respectively. When the API endpoint being written is only meant to perform these basic operations on an object or objects stored in the database, these should be totally sufficient.

This method offers a lot of functionality “out-of-the-box” as compared to the APIInfo method, but because of that is also restrictive. For example, it is not possible to write an endpoint that returns data not encoded as JSON using this method. That’s an uncommon use-case, but not unheard-of.

This method is best used for basic creation, reading, update, and deletion operations performed on simple objects with no structural differences across API versions.

APIInfo

Endpoint handlers can also be defined by simply implementing the net/http.HandlerFunc interface. The net/http.Request reference passed into such handlers provides identifying information for the authenticated user (where applicable) in its context.

To easily obtain the information needed to identify a user and their associated permissions, as well as server configuration information and a database transaction handle, authors should use the github.com/apache/trafficcontrol/traffic_ops/traffic_ops_golang/api.NewInfo function which will return all of that information in a single structure as well as any errors encountered during the process and an appropriate HTTP response code in case of such errors.

This method offers fine control over the endpoint’s logic, but tends to be much more verbose than the endpoints written using the Generic “CRUDer” method. For example, a handler for retrieving an object from the database and returning it to the requesting client encoded as JSON can be twenty or more lines of code, whereas a single call to github.com/apache/trafficcontrol/traffic_ops/traffic_ops_golang/api.GenericCreate provides equivalent functionality.

This method is best used when requests are meant to have extensive side-effects, are performed on unusually structured objects, need fine control of the HTTP headers/options, or operate on objects that have different structures or meanings across API versions.

Extensions

What’s typically meant by “extension” in the context of Traffic Ops is a Check Extensions which provides data for server “checks” which can be viewed in Traffic Portal under Monitor ‣ Cache Checks. This type of extension need not know nor even care which implementation it is being used with, as it interacts with Traffic Ops through the Traffic Ops API.

Traffic Ops supports overrides or new definitions for non-standard Traffic Ops API routes. This type of “extension” is typically reffered to as a “plugin,” and they are described in Go Plugins.

Go Plugins

A plugin is defined by a Go source file in the traffic_ops/traffic_ops_golang/plugin directory, which is expected to be named plugin name.go. A plugin is registered to Traffic Ops by a call to github.com/apache/trafficcontrol/traffic_ops/traffic_ops_golang/plugin.AddPlugin in the source file’s special init function.

A plugin is only enabled at runtime if its name is present in the cdn.conf file’s traffic_ops_golang.plugins array.

Each plugin may also define any, all, or none of the lifecycle hooks provided: load, startup, and onRequest

load: The load function of a plugin, if defined, needs to implement the github.com/apache/trafficcontrol/traffic_ops/traffic_ops_golang/plugin.LoadFunc interface, and will be run when the server starts and after configuration has been loaded. It will be passed the plugins own configuration as it was defined in the cdn.conf file’s traffic_ops_golang.plugin_config map.
onRequest: The onRequest function of a plugin, if defined, needs to implement the github.com/apache/trafficcontrol/traffic_ops/traffic_ops_golang/plugin.OnRequestFunc interface, and will be called on every request made to the Traffic Ops API. Because of this, it’s imperative that the function exit as soon as possible. Note that once one plugin reports that it has served the request, no others will be tried. The order in which plugins are tried is defined by their order in the traffic_ops_golang.plugins array of the cdn.conf configuration file.

See also

It’s very common for this function to behave like a Traffic Ops API endpoint, so when writing a plugin it may be useful to review Writing New Endpoints.
startup: Like load, the startup function of a plugin, if defined, will be called when the server starts and after configuration has been loaded. Unlike load, however, this function should implement the github.com/apache/trafficcontrol/traffic_ops/traffic_ops_golang/plugin.StartupFunc interface and will be passed in the entirety of the server’s configuration, including its own configuration and any shared plugin configuration data as defined in the cdn.conf file’s traffic_ops_golang.plugin_shared_config map.

Example

An example “Hello World” plugin that serves the /_hello request path by just writing “Hello World” in the body of a 200 OK response back to the client is provided in traffic_ops/traffic_ops_golang/plugin/hello_world.go:

package plugin

/*
   Licensed under the Apache License, Version 2.0 (the "License");
   you may not use this file except in compliance with the License.
   You may obtain a copy of the License at

   http://www.apache.org/licenses/LICENSE-2.0

   Unless required by applicable law or agreed to in writing, software
   distributed under the License is distributed on an "AS IS" BASIS,
   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
   See the License for the specific language governing permissions and
   limitations under the License.
*/

import (
    "strings"
)

func init() {
    AddPlugin(10000, Funcs{onRequest: hello}, "example HTTP plugin", "1.0.0")
}

const HelloPath = "/_hello"

func hello(d OnRequestData) IsRequestHandled {
    if !strings.HasPrefix(d.R.URL.Path, HelloPath) {
        return RequestUnhandled
    }
    d.W.Header().Set("Content-Type", "text/plain")
    d.W.Write([]byte("Hello, World!"))
    return RequestHandled
}

Check Extensions

Check Extensions allow you to add custom checks to the Monitor ‣ Cache Checks view.

Extensions are managed using the $TO_HOME/bin/extensions command line script

See also

For more information see Managing Traffic Ops Extensions.

1(1,2,3,4,5,6,7,8,9): The Traffic Ops instance must be using the same PostgreSQL database that the tests will use.
2(1,2,3,4,5,6,7): This does not need to match the name of any pre-existing user.